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PART I 

SELECTING BOOKS FOR ADULT READERS 

ADDRESSED TO ADULT EDUCATORS IN GENERAL 




CHAPTER I . 


INTRODUCTION 

DELINEATION OF THE PROBLEM 

In the spring of 1933 a group of adult educators met for the purpose 
of formulating some of the problems in their field which were most 
in need of studious investigation. Each person at the conference was 
participating in the direction or management of a city or national 
adult education organization and, therefore, was sensible of the de- 
ficiencies of the programs. 

These workers in the field of adult education described many diffi- 
culties j they presented a multitude of unsolved problems. They 
expressed doubts' about the educational philosophy underlying some 
programs, and cited dilemmas growing out of practical exigencies. 
As a result, several areas for research were delineated. 

There were persistent allusions to one phase of adult education, 
allusions which were at first specific and unrelated. Leaders said: 
‘‘Our church groups are tired of the lesson units which we have 
been sending out. How can we make the next unit more inter- 
esting?” Or, “Can any way less wasteful than trial-and-error be 
found to determine the probable success of the material on economics 
which we send to our business groups?” Yet other angles appeared: 
“We have almost stopped suggesting reading references, because 
everything is either too difficult or too childish for our groups. Give 
us something that is simple, yet written for adults.” “We are not 
troubled by vocabulary difficulty, but please don’t let professors 
write for us> They make material so remote from life that students 
refuse to become interested in it.” Each educator’s program involved 
reading material for students in one way or anotherj every use of 
reading appeared to bring, difficulties with it. 

As the various problems came to light and solutions were dis- 
cussed, one common deficiency emerged, namely, the inadequacy of 
present methods of writing or selecting curriculum materials for spe- 
cific adult groups. The immediate need was for curriculum materials 
suited to the particular group of each educator — to groups of textile 
workers, of church women, of college alumni. 

3 
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Two basic and prior needs were admitted, however, which had to 
be filled before the demands of individual educators could be profitably 
supplied. The first was the need to obtain a method which might 
serve as a pattern for each educator who wished to study the reading 
requirements of his adult classes. The second need was for a list of 
standards to judge the adequacy of study materials. The method of 
book selection should indicate practical ways of studying books, and 
of studying groups for whom they were intended. The standards 
should apply to the sort of expository material used in adult classes. 

The discovery of methods for selecting the best reading materials 
for adult study groups was the general problem set by the conference. 
The indicated procedure was, first, to distinguish the standards by 
which different adult groups select reading materials, and, second, to 
analyze those qualities in specified materials that give evidence of 
affecting the judgment of readers. The present study is concerned 
with the solution of the stated problem. 

PLAN OF THE BOOK 

Part I of this study is prepared for the general reader, for the 
educator whose interest lies chiefly in the practical recommendations. 
It includes an investigation of pertinent practices in adult education 
and introduces an experiment in book selection. Part II contains 
the technical aspects of the solution of the problem. The details of 
this research study are given in order to provide the information 
necessary to evaluate the experiment and to apply any phase of it to 
other situations. The reader of Part I who desires to find substantia- 
tion or elaboration of particular statements will be guided by refer- 
ences to Part II. 

Part I sketches the status of program planning in this area among 
adult educators. The survey shows the manner of selecting reading 
material which was found in a cross section of programs, demonstrates 
the values that have been recognized in careful curriculum planning, 
and describes methods that have been found practicable in selecting 
and evaluating the curriculum. In describing these methods of cur- 
riculum selection, the lack of adequate techniques for supplying groups 
with the proper literature is recounted. 

In addition. Part I describes an experiment with a method of 
studying an adult group’s tastes for selected non-fiction material. 
The experiment furnishes bases for deriving standards of judging 
non-fiction. At its conclusion, Part I recommends subjects for fur- 
ther investigation. 
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Part II states the techniques used in discovering and classifying 
the data upon which criteria may be based. The description of 
methods, both of analyzing the books used in this experiment and of 
studying the group that read the books, provides guideposts to other 
students of adult curriculum materials. The final chapter gathers all 
of the experimental evidence both from research literature and from 
this study for the purpose of making a definitive description of 
each of the standards for judging non-fiction reading material which 
were found in the experiment. 

PROCEDURE FOR SOLVING THE PROBLEM 
In solving the problem of procuring methods and standards for 
selecting study materials, a survey of pertinent literature — research 
studies, statements of the philosophy and of the practice of adult 
education — ^was first made. Then an experiment was conducted to 
develop a methodology for setting up standards for a single type of 
literature for a particular type of person. 

The subjects for the experiment were a group of forty-five men 
and women employed under the Civil Works Administration in the 
spring of 1934. Every subject read three pamphlets on each of four 
topics dealing with unemployment, war and peace, changing govern- 
ments, and unstable money. The reader was expected to write 
criticisms of each pamphlet in accordance with mimeographed in- 
structions. Then he rated each section of the pamphlet and stated 
his reason for the degree of interest expressed by the rating 5 he 
made a general criticism of the pamphlet and gave it a i, 2, or 3 
rating according to his interest in it. Finally, the reader compared 
the three pamphlets on each topic with one another. 

In the meantime, several types of analysis were being carried for- 
ward by the author and several assistants, who were selected on the 
basis of demonstrated ability from among the Civil Works Adminis- 
tration group. Information about the group of readers was collected 
and tabulated; certain qualities of the experimental pamphlets were 
studied; and the criticisms of the readers were analyzed as they sub- 
mitted their reports on pamphlets. 

Each subject filled in a questionnaire about his education, occupa- 
tion, club affiliations, and reading interests. After the answers to 
each question were tabulated, those facts about the group that showed 
clear-cut differences were selected for further study. On the basis of 
this information sub-groups with different backgrounds were formed, 
and their opinions were examined to see whether their preferences 
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for pamphlets and their types of criticism differed. Of the six con- 
trasted sub-groups which are reported, two differed in sex, two in 
education, and two in social science training. 

In addition to studying facts about the group, qualities of the 
pamphlets were investigated. Assistants made a resum6 of the con- 
tent of each pamphlet, described the format, and listed methods used 
by authors to catch and to sustain the reader’s interest. They 
analyzed the difficulty of each pamphlet with the aid of a chart de- 
signed to help tabulate words and sentence structure and essayed the 
quantity of history and the quantity of figures by counting the fre- 
quency of each in the pamphlets. Three of the assistants attempted to 
judge the pamphlets upon literary merit and upon author’s bias in 
the presentation of controversial information. In order to sum- 
marize the collected information, the pamphlets were ranked in their 
order of difficulty, of quantity of figures, and of quantity of history j 
and a numerical description of the author’s bias was made. The other 
analyses performed by the assistants remained in paragraph form. 

The readers’ criticisms of the pamphlets were utilized in various 
ways. The criticisms indicated the standards which these readers used 
in judging pamphlets. They also yielded a Pamphlet Preference 
Index which could be compared with the ranked qualities of the 
pamphlets. The opinions provided additional information about 
the nature of the pamphlets. Finally, they were used to reveal 
reasons for differences in preferences among the sub-groups. 

All of the information about the nature of the experimental group, 
the qualities of the experimental pamphlets, and the opinions of the 
group about the pamphlets was assembled for one main purpose. The 
sum total of this information aided in collecting and describing a list 
of thirty-eight standards^ for judging non-fiction material. The 
standards, which are grounded on the experimental evidence offered 
from the analysis of pamphlets and the study of a group and their 
opinions, provide a more objective method than heretofore available 
for selecting books on current controversial issues. 

UNDERLYING SCIENTIFIC PHILOSOPHY 

Since the method employed in the approach to the problem of 
determining criteria for selecting reading materials is relatively new 
in its application to educational problems, a brief discussion of the 
philosophy and resultant techniques is necessary. 

The physical sciences of sixty or forty years ago set a pattern 
’■A list of the standards is g-iven in Chap. IV, Table V, pp. 82-83. 
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of experimentation and did not take due cognizance of interrelation- 
ships and of change. The physical sciences have long since re- 
linquished that pattern, but the social sciences, with their culmination 
in the mechanistic beliefs of behavioristic psychology, are just now 
beginning to break away from it. 

The methods which are being developed by the social sciences are 
based upon concepts of change, of growth, and of interaction. They 
were formulated to some extent by Gestalt, or organismic, psy- 
chology, which in turn follows discoveries of the new physics. Gestalt 
psychology points out that the whole is different from the sum of its 
parts, that a totality has unique properties different from each of its 
elements taken separately. This concept requires as wide a considera- 
tion of the ramifications of a problem as possible. Correlative to the 
thesis of the importance of wholeness is emphasis upon the importance 
of the interrelationships of elements with one another. A stick of 
dynamite and a fuse may be studied, dissected, and described when 
separate, but no description of the two separately could convey an 
adequate impression of them when together and interacting. 

Motivated by the principles stated above, the scientists of human 
life — ^psychologists, sociologists, educators — ^have searched for meth- 
ods of describing large units of behavior in terms of many of the 
elements that contribute to that behavior rather than of one frag- 
ment isolated for inspection. One of the most popular devices of 
the sociologist is the case method, in which all pertinent available 
information, derived from both tests and records and from opinions 
of those familiar with the problem, is assembled to show the effect 
of interacting phenomena. Sociologists are also mapping trends, dis- 
covering the usual outcomes of a set of conditions through a com- 
posite examination of many case studies. 

The present study is an experiment which utilizes some of these 
newer scientific techniques. It represents one observation among 
many that are needed. Within the limits of the data at hand, how- 
ever, the perspective of wholeness, within which interrelations are 
important, was kept. In brief, the problem was not the determina- 
tion of a simple linear cause and effect relationship but a necessarily 
less precise, because more complex, exploration of multiple interre- 
lationships. 

In working out the principle of an integrated total view of the 
problem, the whole field was considered: the relation of a curriculum 
to adult education, the place of reading in a program, and the need 
for better methods of selecting reading materials. The experimental 
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aspect incorporated a comprehensive approach. Readers were in- 
structed to give general as well as analytical impressions of the books 
they read. Elements were isolated only after study as parts of the 
total setting. Their interrelationships were kept in mind, even while 
the qualities of each were noted separately. 

Sociological methods were employed. The case study technique 
was utilized and showed distinct differences between the most popu- 
lar and the least liked of the reading material. This method con- 
tributed data that could not be obtained in any other way. In order 
to observe general trends, information was massed. Such a problem 
as the following was set; Given twelve different books introducing 
all shades of propaganda, and evaluated by forty-five different people, 
which shade of propaganda did the readers like best.? Another prob- 
lem employing the technique of massing information was: Out of 
thousands of reasons given for liking or disliking books, which were 
most often mentioned.? 

To further the intention of studying as many interrelations in a 
unit of behavior as possible, this experiment was limited to the careful 
analysis of the behavior of a small number of readers who examined 
only twelve pamphlets. However, just as psychoJogists may acquire 
a knowledge of the laws of behavior and physicians may learn the 
nature of a disease from observing a single patient, so we hoped to 
discover some boundaries of reader judgments through the study 
of a single adult group. The findings, therefore, are not statistically 
significant j but they are important not only in providing information 
about the experimental group and in indicating a methodology for 
analyzing other small groups, but also in suggesting the character- 
istics of a much broader field, 

LIMITATIONS OF THE STUDY 

The present experimental study of factors causing interest in read- 
ing has two chief limitations. Its findings are restricted and tentative. 
And the method is not directly applicable to the needs of other 
groups. 

In regard to the limitations of the findings, a study is rarely able 
to explore a new field and at the same time to establish definite con- 
clusions regarding the details of its subject. This investigation in- 
troduced a technique and collected all available information from 
previous research on standards for selecting reading material; but 
definite conclusions about the standards must be formulated by other 
workers. This study represents the observation of one group, which 
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read a single type of literature, whereas similar reports upon the 
interests in various topics of many groups 'are needed. The present 
study names thirty-eight standards which are logical areas of inves- 
tigation, among which only seven areas were inspected. There 
should be intensive studies of all the criteria, both separately and in 
relation to other criteria. 

Second among the major limitations, the experimental techniques 
may appear impractical to adult educators interested in solving prob- 
lems of curriculum selection. In the present experiment, forty-five 
readers were paid to spend a week making a detailed analysis of 
twelve booklets. A repetition of this thorough study will not be 
possible for those experimenters who must deal with volunteer assist- 
ance in determining the interest of reading materials. Suggestions 
are made in Chapter III for adapting the techniques used in this 
study to other experimental problems and conditions. 

ASSUMPTIONS UPON WHICH THE STUDY RESTS 

The viewpoint which animated the study of methods for determin- 
ing the best reading matter for adult study groups is summarized in 
the following six points. 

1. Modern society has problems and conflicts which threaten the 
foundation of old philosophies and habits. If democracy is to func- 
tion successfully, it is essential that adults become intelligently in- 
formed upon these problems. 

2. The issues involved in modern problems must be presented to 
adults in such ways as to hold their interest, both because the best 
learning takes place under such conditions, and also because that is 
the only method through which large numbers of people will be en- 
listed in the study of these crucial problems. 

3. Although many means of communication must be employed in 
education for modern living, one of the chief tools is reading. There- 
fore, we must learn how to make reading material interesting, as 
well as profitable, to all citizens. 

4. In many types of experimentation the opinion of the experi- 
mental subject is not investigated. But when the object of a study 
is the selection of books that will be interesting to specific groups, 
the expressed opinions of typical members of those groups should be 
accepted as an important basis for the analysis of books. Even though 
additional unmentioned factors may be considered by the research 
worker, the verbalized opinions of group members are the only data 
which they can supply, and therefore are valuable. 
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5. Although some adult educators believe that in general scien- 
tific methods are invalid in their field, we hold that a scientific ap- 
proach to the curriculum problems of adult education is both desir- 
able and feasible. 

6. A scientific investigation of reading standards must recognize 
the interdependence of various factors upon one another. In addition, 
as much as possible must be learned about each factor individually. 
Since the aim of an experiment is prediction, analysis of factors sepa- 
rately and in relationship will lead to that end better than an exami- 
nation restricted to complete units of subject matter or behavior. 
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THE CURRICULUM IN ADULT EDUCATION 

The problem of this book, as outlined in the first chapter, is the dis- 
covery of methods of selecting the best reading materials for adults. 
The mere statement of the problem raises several doubts which 
must be dispelled before approaching the problem’s solution. 

Is a curriculum essential to an adult education activity? Can 
group members be expected to read.? What methods are the organi- 
zations of adult education now using in the selection and evaluation 
of curriculum materials? Are there any practical ways in which the 
selection and evaluation of materials may be improved.? 

VALUES OF A CURRICULUM IN ADULT EDUCATION 

Adult educators do not often use the word “curriculum”; on the 
whole, the ideas of the profession are not clearly formulated in 
regard to it. For the purposes of discussion, however, let us agree 
on a working definition. The curriculum in adult education is a 
flanned frogram of study y set uf with methods and materials indi- 
cated for the fulfillment of definite aims. According to this definition, 
a curriculum — ^which is only one of the tools of a sound program of 
adult education— demands several things. 

In the first place, a curriculum demands that a group which is par- 
ticipating in a program gather for the serious purpose of inquiring, 
“What kind of an education are we trying to acquire.? What are 
the aims of our program?” Although the educative process itself 
may modify the aims that a group sets, the occasional formulation of 
goals should lend purpose and continuity to a program. Secondly, a 
planned curriculum necessitates discovering those possible activities 
and materials which will aid in reaching the goals. The demands 
of group members for a rich content of courses should be met better 
than ever before. Thirdly, the methods and materials which were 
introduced in order to fulfill the original plans and goals must be 
evaluated to determine whether they actually produced the intended 
results. The periodic evaluation of programs will indicate which 
methods and materials were practical and which should be improved. 
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Not all adult activities need a curricvilum in the formal sense of the 
word. But an informal consideration of the three principles enum- 
erated would benefit all adults engaged in group activities. Applica- 
tion of the requirement, that goals and purposes be discussed and 
formulated, would clarify the function of each organization in the 
minds of its members. Many groups which until now have had little 
in the way of a planned program would profit by preparing reason- 
able activities and materials to lead toward the goals. All groups 
that consciously strive to find the best activities and materials should 
welcome a critical evaluation which will show them how to fulfill the 
goals more adequately. 

READING AS A NECESSARY PROCESS IN GROUP STUDY 

People have long been accustomed to absorb information passively 
through lectures or reading. Many of them have not yet realized 
how little they learn that can be applied to their problems when 
they sit still and let words bombard them. But they must surely dis- 
cover before long that, in order to attain the Worth-while satisfac- 
tions which they expect from study groups and other agencies, they 
must expend some effort themselves. 

In the creative fields, participation in group activities has become 
a fact. Ladies and gentlemen who listened politely to cultured lec- 
turers discuss art in the last century are superseded by women — and 
men — ^with sticky fingers and paint-daubed faces, who are thoroughly ’ 
enjoying themselves while learning about form and color through 
the medium of paint or clay. 

The move for taking part in the discussion of social questions has 
not kept pace with discoveries in the field of play. Although people 
are prone to talk about public problems, most of them have not 
searched for materials to help solve the problems. They are not 
yet beyond the stage of asking equally ignorant neighbors for opin- 
ions or of holding as sacred the insight of the cartoonist in the daily 
paper. They are still like the art students of an earlier day who 
listened to lectures or talked about pictures with no experience in 
accumulating information of their own with which to evaluate the 
lectures or the pictures. 

When citizens are ready to seek intelligent answers to social prob- 
lems, many activities must contribute to furnishing the facts and 
awakening a state of mind which will enable them to draw reasoned 
conclusions. Lectures, trips, exhibits — all have their place in enabling 
people to understand the subjects in which they are interested. 
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Most important of the information-yielding mediums is reading. 
The study of a problem through the words of thoughtful writers 
gives a richer background more efficiently than does any other method. 
Written records do not share many of the handicaps of the less per- 
manent agencies of platform, microphone, and movie reel. Litera- 
ture provides detailed and definite information. It must guard its 
accuracy, because readers may easily return and check it. It can be 
studied and kept for reference. 

So essential, indeed, is the ability to give or to gain information 
through the medium of the written page that one of the chief func- 
tions of the schools is the teaching of reading} and by far the greatest 
concentration of effort of those who wish to reach the public is upon 
preparation of printed matter. 

Many surveys, as well as the practical experience of those working 
with adults, have demonstrated the fact that most people do not 
sufficiently utilize the facilities of reading to increase their informa- 
tion. Whether the failure to learn adequate reading techniques may 
be laid at the door of the schools or elsewhere, it is evident that one 
great weakness in the equipment of the majority of people (college 
graduates as well as elementary school graduates) is the inability 
to extract a full meaning from the printed pa'ge and to enjoy the 
process. 

There are two principal reasons for these inadequacies in reading. 
One is deficiency in the skill of reading. The other is a lack of 
interest due to a training which failed to demonstrate the practical 
value of books. A large number of people literally do not know 
how to read in any meaningful sense. They do not have the vocabu- 
lary} it is hard for them to pick out important points in involved 
paragraphs} material without human interest elements (i.e., not di- 
rectly related to their interests) seems dull and wordy. An editorial 
in The New Refublic [4]* several years ago answered critics who 
claimed that the tabloid newspapers, which contain simplified con- 
cepts and a dramatized style, were luring readers from the other 
papers. The magazine editor had gathered statistical evidence to 
show that “the tabloids marshal a whole new army of readers of 
their own.” The writer added the hypothesis that through the guid- 
ance of the tabloids this army is gradually learning to read the more 
complex newspapers. 

Although most study groups may not be composed of tabloid 

* Throughout this chapter numbers in brackets refer to bibliographical leferences 
to Chapter 11, as given on pages 32-33. 
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readers, the members aJl too frequently do not have the background 
of concepts and skills necessary to read the books that are ordinarily 
recommended. Their normal reading interests are satisfied by The 
Saturday Evening Post and Gene Stratton Porter rather than by 
HarfePs and Ellen Glasgow. 

Many group members who have the ability to grasp most of the 
reading suggested by their leader are not interested in following 
his suggestions. It would appear that the conditioning of past years, 
when reading often took the form of dull assignments upon topics 
of no apparent significance, had caused them to rebel against any- 
thing which looked like an assignment. A tactful but determined 
group leader may plan a long-term campaign in which this disagree- 
able mind-set is overcome. Covert allusions to fascinating material 
on their level and occasional requests for reports on brief and easily 
obtained articles may lay the foundation for discussion based on 
articles, later on books, and finally on problems which demand infor- 
mation from many sources. 

There are two alternative, or two parallel, procedures for adult 
education in reading techniques. On the one hand, leaders can 
teach students to read better j on the other hand, they can give them 
the type of material that they already like to read. 

Several courses in remedial reading have been offered to adults, 
chiefly to college students. The grateful testimonials [6] written 
by students who entered the classes unwillingly and remained to re- 
joice in their improved efficiency suggest the potential demand for a 
course in Efficient Reading.^ The tremendous success of public 
speaking classes which aim to overcome deficiencies in speech presents 
an encouraging parallel. 

When it is not practical to offer a course designed to improve 
reading habits, every leader can encourage timid readers by giving 
them brief, simple, and interesting books or articles on the topics 
that the group is discussing. The first source of information about 
appropriate material may well be the Readers’ Advisory Service, 
developed by librarians in forty urban libraries. Current magazine 
articles are readily discovered through the Readers^ Guide to Peri- 
odical Literature, which is on the reference table in every library. 
The best pamphlets on many topics were selected in 1934 by the Read- 
ers’ Advisory Office of the Cleveland Public Library [3], in a 
booklet entitled Readable Pamfhlets. The bibliography. Books of 

’■ Such a course was reported at the Enoch Pratt Free Library of Baltimore, Mary- 
land. [j] 
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Genercil Interest for Today’s Readers [9], classifies books under 
many headings and contains delightful annotations. Each of the 
latter sources indicates the easiest treatment of each topic. With 
such aids as these, groups can be led to read and the scope of the 
selections can be broadened as the members’ habits become established. 

PRESENT STATUS OF THE SELECTION OF CURRICULUM MATERIALS 

The preceding pages have stated that most adult educational organi- 
zations need to formulate goals and then to provide abundant ma- 
terials and opportunity for activities which would aid the partici- 
pants in reaching the goals. The thesis has been set forth that reading 
materials can contribute vitally to programs provided group members 
are able to read and to become interested in the content of the litera- 
ture. In technical educational vocabulary such literature is known 
as curriculum material. 

This study does not cover the whole range of curriculum problems 
in adult education. Nor can it apply to more than one field the 
technique later proposed for selecting reading materials. But the 
whole status of adult education curricula and the whole problem of 
selection of materials are so very new that a review of the present 
status of the selection of curriculum materials seems in order before 
proceeding to the main problem of this study. 

In passing, it may be noted that behind the problem of selecting 
materials is the equally knotty task of producing books and other 
printed matter that are readable. After the data gathering of this 
study was completed, a research project in readability, financed by 
a foundation grant, was set up at Teachers College, Columbia 
University. 

Many adult programs incorporate printed material in their study 
plans. It appears desirable and pertinent to know what provisions 
educational organizations now make for selecting materials that are 
within the ability and interest range of their study groups. A cor- 
relative topic for investigation was the ascertainment of methods used 
by organizations to evaluate^ the success of their recommended 
material. 

In order to investigate these questions, William Kolodney, an 
adult education administrator and a student at Teachers College, has 

° In order to avoid confusion of thought, it should be pointed out that selection of 
new material and evaluation of old are actually part of the same circular process; we 
evaluate now in order, that selection may be improved next times or we evaluate in 
order to modify the criticized material before it is finally selected for -wide use. 
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permitted me to examine and to report upon the responses to a letter 
which he sent out in the spnng of 1934.® H received replies to 
sixty letters from forty-seven prominent adult educators throughout 
the country concerning the selection of the curriculum material that 
they used and their methods of evaluating its effectiveness. 

Twenty-eight of the adult educators who responded to Mr. Kolod- 
ney’s letter were using written curriculum materials.^ Twelve of 
them stated that materials were prepared by the group directing the 
programs at either national or loc^ headquarters. Nine said that 
instructors prepared the written content of their courses of study. 
Seven educators utilized printed material. 

The letters contained many specific suggestions for obtaining cur- 
riculum content, some of which will be reported.® Two examples 
of the preparation of materials at national headquarters by the group 
directing the program will be cited. Also, two methods-o£ preparing 
books with the aid of a class will be illustrated. Then a procedure 
for selection of printed material will be stated. The first illustration 
of the preparation of materials at headquarters follows. 

The National Federation of Business and Professional Women’s 
Clubs set as a ten-year objective the study of economics, especially in 
its relation to the individual woman and her community. With this 
major aim agreed upon, curriculum materials were prepared by ex- 
perts under the guidance of the educational director, Frances Cum- 
mings, and were distributed to the thirteen hundred clubs in response 
to demands for material upon certain topics. 

A common practice was stated by Grace Coyle, at that time ex- 
ecutive head of the Laboratory Division of the Young Women’s 
Christian Association, who said in a letter, “In some cases when we 
are preparing material here at headquarters we send it out to a 
selected group of local secretaries for use and criticism before it is 
finally published.” 

S. M. Keeny, secretary of the Association Press of the Young 
Men’s Christian Association, described a method of preparing a book 
which grew out of practical experience with groups. His letter to 
Mr. Kolodney is quoted: 

® TI>e letter is given in Appendix II E. 

* Nineteen of the forty-seven adult educators who responded were in a supervisory 
capacity where no courses were offered. 

“ On the basis of information suggested in the letters, each of the persons named in 
this section of the report, except P. N, Youtz, was interviewed by the writer. 
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Much of the material which we use in discussion groups or formal classes 
is developed experimentally by working along with those interested in the 
problem. 

For example, in working out one book an effort was made to discover those 
local associations where leaders were meeting together for preparation of 
materials, in this case for Bible study. These weekly meetings were used as 
the centers through which problems were isolated and outlines prepared. 
Each week the teachers of boys’ groups came together to prepare the outline 
for a lesson on a topic in which the members of their groups were interested. 
After preparation, each leader went before his own group with the outline, 
trying it out on the members, and reporting its success the next week to the 
leaders’ group. The completed outlines from these experiments were sent 
to an editing group, who put the material into the form in which it finally 
appeared in publication. 

A more direct approach for study group collaboration in preparing 
materials is described in an article by P. N. Youtz [18], in which a 
book grows out of interchange of ideas between an author and his 
discussion group. 

Adults — well informed, intelligent, serious readers — ^are brought together 
in groups of twenty-five to one hundred to work out books. The author brings 
the central concepts and tries them out on the people, modifying and adapt- 
ing them to fit the needs of the group. This is written in book form and 
published. 

The New York Public Library and the People’s Institute cooperate in 
bringing the authors in, paying them for coming. The discussion is led by 
a trained, artistic leader. 

The Child Study Association has a committee for the selection of 
books, which was described to the author by Marion Miller, education 
secretary of the Association. It is called the Committee on Bibliog- 
raphy and has about thirty members, all of whom are experienced in 
parent education or in the field of parent education literature. The 
c6mmlttee holds periodic meetings at which three written reviews 
of each book under discussion are read. The reviews are subjected 
to critical examination by this committee of experts. If the book is 
considered inaccurate, trivial, or unsuited to the needs of parents. 
It Is rejected. If it is thought worth while, it may be placed on a 
reading list for leaders or for their study groups. The list is revised 
yearly. Its content furnishes recommendations for the curriculum 
of child study groups throughout the country. 

Many methods of evaluating the success of curriculum materials 
were stated by the adult educators who responded to William Kolod- 
ney’s letter. They are summarized in Table I. 
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Table I 

Methods of Evaluating Curriculum Material 
as Described by 25 Adult Educators 


Method 


Number 

Stating 

Method 


I. In the qualitative judgment of success of materials, the data 
employed were: 

A. Student opinion 4 

B. Instructor opinion (no stated basis for judgment) . , 5 

C. Instructor opinion (specific factors cited were) : 

I . Amount and quality of discussion 3 

a. Quality of students’ work in exhibit 1 

3. Attainment of general goals, namely: 

a. Changed living .... a 

b. Widened horizons ... 1 

c. Culture i 

d. “Results” I 

11 . In the quantitative measurement of materials, the instru- 
ments used were: 

A. Attendance count . . . 3 

B. Tabulation of students’ answers to questionnaire ... i 

C. Response on examination . . i 

D. Number of books sold i 

E. Growth demonstrated on rating scale i 


When adult educators were asked how they decided which cur- 
riculum materials were successful with their patrons, they frequently 
replied in general terms. Nine said merely that they obtained the 
opinion of students or of instructors upon the materials. A few edu- 
cators were more specific and gave some of the factors that deter- 
mined instructors’ judgments. These factors were the quality and 
amount of discussion stimulated m class, the quality of workmanship 
which the course of study inspired, and the extent to which certain 
general goals of adult education were achieved. 

The quantitative measurements of success listed under II in Table 
I in most instances deal with the success of a program or a course, 
rather than with the success of study materials. The criterion of 
large or small attendance is frequently applied, but its weakness as 
a measurement of the value of courses is admitted by those who 
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sponsor programs. Likewise, the criterion of the number of books 
sold is open to the objection that a large or small number is slight 
indication of the value of the book to the reader. The questionnaire 
which was mentioned by one educator dealt with the broad outcomes 
gained from attendance at the adult school, rather than with the 
narrow outcomes of particular courses or books. The only two meth- 
ods listed which might show the specific value of curriculum ma- 
terials were the use of rating scales for checking progress and ex- 
aminations. 

These methods of evaluation are the expression of pioneers in a 
new field who are close to their work and who believe that the in- 
tangible outcomes from education are frequently the most vital. 
Most of these leaders would agree with the last quotation below, 
saying that it is impossible to isolate one factor and to maintain that 
it is the cause of growth in a student. They believe rather that all 
the educational processes, proceeding simultaneously, produce a more 
or less immeasurable improvement. 

The following quotations reflect attitudes expressed in the letters 
concerning methods of evaluating the success of courses. 

In preparing material or in advising on programs, we lay particular stress 
on meeting study groups on their own ground, and on making the study 
process one of extending or deepening existing interests rather than of in- 
troducing interests entirely alien to the members. . . . The measure of suc- 
cess is the extent to which the study outline and the material provided with 
it succeed in arousing responsible interest} and that means an interest that 
links itself to the situations which members of the course have to meet in 
their own lives. 

In regard to determining the success of the books we use, our judgment is 
based largely on the quantity and quality of discussion which the reading 
provokes. In groups as small as ours, it is possible to watch carefully the indi- 
vidual reactions. 

The greater majority of our students remain with us for several years. It 
is our judgment that the important result of these courses is not merely in- 
formation, but a process of reorientation. We have from time to time in 
the past distributed questionnaires, and the returns always show this result 
to be uppermost. 

We subject ourselves to constant criticism. If at the end of a class I feel I 
have failed to hold the interest of the young people, I try to analyze the rea- 
sons for this. In my own case it is usually because I fly too high and get 
away from connection with the young people’s experience. The other mem- 
bers of our group would, I think, give the same answer. 
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The final evaluation must come in the awakening of the student. As one 
may become interested through woodwork, another through geography or 
history, and another through singing games, gymnastics, and so forth, it is a 
little hard for the individual teacher to know what part his courses have had 
in the end results. My own feeling is that such success as we have is due to 
the many channels through which we reach the individual, and most especially 
through the “home life and relationship of equals” so emphasized by the Danes. 
It will take a generation to show what the effect of such teaching, plus our 
effort to build up economic life, will be in this section. 

Although the letter addressed by William Kolodney to the adult 
educators asked for information concerning their methods of evaluat- 
ing curriculum materials, the answers were couched in terms of 
evaluation of the whole adult program. It is probable that the 
writers had used no specific devices which they could describe. The 
persons who responded are among the leaders in adult education in 
the nation and possibly represent superior practices. Even if the re- 
ports of these educators are merely considered typical, they indicate 
that methods for determining the effect of curriculum materials upon 
readers have not been formulated or practiced generally in adult 
education. 

The spirit of adult education as shown in these quotations is in- 
valuable in any interpretation of the results of programs. The firm 
insistence upon recognition of the value of changing the attitudes and 
increasing the joy of students, together with the realization that a 
combination of factors contributes to the desired results, should guide 
any attempts at appraisal. But this general sort of appraisal is not 
enough; the agencies that finance adult education organizations will 
demand tangible evidences of specific benefits derived from its pro- 
grams. Directors, too, will need proof of the success of some of their 
methods and of the failure of others. Separate evaluations of distinct 
phases of the program should be planned. For instance, methods of 
evaluating the phase with which this study is concerned, that of the 
curriculum, should be evolved. Some techniques that have been 
worked out by adult educators will be suggested. 

The evaluation of a program is the last step of a process which 
must be viewed in its entirety in order to be judged fairly. The 
evaluation of the success of reading materials is dependent upon these 
processes: ascertaining the purposes of the program, recognizing the 
needs of participants, and supplying literature that meets certain spe- 
cific requirements. Recommendations for improving the curriculum 
in these areas will be given. 
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RECOMMENDATIONS EOR OBTAINING A CURRICULUM FOR AN ADULT 
EDUCATION PROJECT 

The meaning of the term curriculum., as previously defined, in- 
cludes much more than mere reading matter in a course. In order 
to obtain a curriculum, it is necessary for an adult education organi- 
zation to erect goals, to make a wise selection of reading materials, 
and then to evaluate the materials and the organization’s progress 
toward attaining the goals. Another step should be included if i 
proper goals and proper materials are to be selected: the leaders of 
the organization should learn as much as possible about the com- 
munity and the groups who will pursue the program. 

Recommendations for following the steps incident to obtaining 
an adequate curriculum for an adult education organization or study 
group will be discussed in the following pages under these heads: 

A. Determination of the goals for an adult education program. 

B. Study of the community and the group for whom the program is intended. 

1. Survey for the purpose of gaining general information about the group 
or community. 

2. Survey to detect the interest of adults in activities or in subjects. 

3. Tests to ascertain various abilities of gioup members. 

4. Analysis of shortages in the lives of group members. 

C. Selection of study materials. 

1. Methods of writing materials. 

2. Methods of selecting printed materials. 

D. Evaluation. 

1. Evaluation of program. Discovering changes in habits; use of interviews, 
diaries, check lists; records of projects. 

2. Evaluation of materials: Testing the effect of materials on students; uti- 
lizing the leader’s opinion; utilizing the study group’s opinion. 

These topics will be treated in order. 

A. Determination of the Goals for an Adult Education Program 
The first section of this chapter emphasized the need for working 
out definite plans and goals for whole organizations and for each 
group within the organization. Every group that meets should have 
its energies focused in a direction which the members agree is worthy. 
Goals should be a living, motivating part of all work. And they 
should be revised as purposes change or grow. The following illus- 
tration shows how one organization utilizes its resources in planning 
its programs. 
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Example: 

An integral part of the Young Women’s Christian Association program is its 
planning committees. Purposes are formulated in small groups, are carried 
to larger groups, and after reaching the attention of the national organization, 
are carried back to enrich the programs of all member branches. 

At the Washington, D. C., branch of the Y.W.C.A.,® each study group or 
club elects a representative to meet other representatives in the department. 
In the spring all of the representatives discuss the work of the past term, make 
suggestions for improving the work or for adding other types of activities, 
and map out goals and methods for the following term. 

Ideas worked out by the local Y.W.C.A. departments of Girl Reserves, 
business and professional clubs, education, etc., are pooled yearly at regional 
conferences. There are biennial conventions at which the best ideas, devel- 
oped by all of the departments over a two-year period, are discussed. At the 
summer convention a survey of progress is taken, aims are re-evaluated and 
re-interpreted, and methods of procedure are developed. 

In the fall of the year aU of the people at the Washington branch of the 
Y.W.C.A. who act in any administrative capacity — ^as group leaders, depart- 
ment directors, active workers — ^meet in general conference. The recom- 
mendations of the regional or national convention are presented. After several 
days’ conference upon general and specific purposes, the various departments 
proceed to complete the organization of theii work. 

Any department’s program, then, will be based upon three sources: (a) the 
previous year’s experience, including recommendations from group representa- 
tives, (b) the general purposes of the organization, which were derived both 
from contact with other branches and from the integrating meeting in the 
Washington branch, (c) the purposes and interests of the new groups, as they 
are formulated during the term. 

B. Study of the Conwmmty and the Grouf for Whom the Program 

Is Intended 

Since any group is part of a neighborhood or community, the col- 
lection of adequate knowledge about the group’s patterns of thought 
and action brings the leader of even a single group to see the de- 
sirability of studying its community background of work and recrea- 
tion. 

An educational program should be based upon the interests of 
group members, upon their intellectual achievement and ability, upon 
the previous experiences that provide materials for enrichment of 
concepts, and upon their immediate needs. 

No prescription for becoming acquainted with a group can be 

" Interview with Nettie Anderson, General Secretary, Y W.C.A., Washington, D. C., 
February 7, 1935. 
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made. The examples given here draw much of their meaning from 
the fact that they grew out of real situations in answer to problems 
that needed solving. The methods are important and valid when, 
and only when, they are part of a total educative setting. The fol- 
lowing examples are suggestions rather than patterns. 

1. Survey for the furfose of gaining genet ai information about 
the grouf or community 

Most leaders gather general information about the education, age, 
occupation, marital status, sex, and interests in reading or other per- 
tinent activities of the group. There are sometimes occasions for 
collecting more extensive information than this 5 there are frequently 
opportunities for making greater use of the information already col- 
lected than is appreciated. 

Examples: 

In this study the usual information collected about the experimental group 
was used as one of the chief keys to the solution of the problem of reading 
standards. (See Chap. V, pp. 137-41 and Appendix II E.) 

Methods of making a thorough community study are amply illustrated in 
“Middletown.” [12] Extensive data were gathered under the general topical 
headings, “Getting a Living,” “Making a Home,” “Training the Young,” 
“Using Leisure,” “Engaging in Religious Practices,” and “Engaging in Com- 
munity Activities.” 

In order to write a book upon the methodology of social investigation, 
Luther Fry [7] studied all of the literature upon community surveys. The 
book contains an extensive bibliography, as well as information about planning 
a study, consulting available sources, observation, the interview, questionnaire, 
analysis of findings and interpretation to the community. 

2. Survey to detect the interest of adults in activities or in subjects 
A paradoxical problem presents itself when directors try to base 

programs upon the interests of patrons. They wish to find interests, 
because they believe that any program which runs counter to interests, 
or which is injected where there is no trace of interests, will probably 
fail. On the other hand, they can find few well-defined interests 
upon which to build a program. The solution to the paradox lies 
in ascertaining patrons’ prejudices against some activities and their 
indications of latent interest in others. At the same time, other 
evidence than expressed interests must be studied to show which 
activities would probably be successful with groups. Various meth- 
ods for ascertaining the interest of adults are used. 
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Example: 

Tlie Radburn Plan [lo] is based upon the satisfying of community de- 
mands. A questionnaire was sent to community residents, asking them which 
subjects they would like to study and which outdoor and indoor recreational 
activities they would enjoy. A program was built on the basis of expressed 
interests. Careful checks designed to indicate the response to the offerings 
were made. When more demands arose, new recreational opportunities were 
opened. 

As a supplement to the questionnaire method of plumbing interest, “word 
of mouth” was relied upon to furnish further guides to selection. As Robert 
B. Hudson expresses it, “The results of the questionnaire were submitted to 
the committee concerned. On the basis of the interest its members may have 
heard expressed by residents, plus the amount of interest expressed on the 
questionnaire, the committee selected the activities to be presented.” [lo, 
P- 83-] 

3. Tests to ascertain various abilities of grouf members 
The public school system has demonstrated that an immense 
amount of human waste can be avoided if accurate information about 
pupils’ abilities is obtained. The schools administer intelligence tests 
that enable the placing of children in surroundings most congenial 
and worth-while for them as individuals. They give reading tests that 
show teachers how to help children to adjust to an important aspect 
of their educational environment. The schools also administer sub- 
ject matter tests that immediately reveal weaknesses and points of 
advancement which teachers must consider. Furthermore, they work 
out methods of studying children’s emotional natures in order to aid 
them in well-rounded development. Similar estimates of the ability 
of adults may prove valuable. 

Example: 

During the economic emergency it has been possible to conduct experiments 
with unemployed adults in ways which should produce suggestive results foi 
all administrators. Major centers have been established in Minneapolis and 
in New York City which have tried out all of the known methods for studying 
the abilities of adults in order to recommend better vocational and recreational 
adjustment for them [i] From this elaborate testing have come some practi- 
cal recommendations for methods of discovering and utilizing abilities. 

While it is not recommended at the present time that adult edu- 
cators give as many tests to their students as do either the public 
schools or the research centers, estimates of some of the abilities and 
aptitudes of adult students should be made. Whenever possible, a 
few simple tests, showing reading ability in particular and background 
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of knowledge in general, should be given. Facilities should be pres- 
ent for administering intelligence tests when students express an 
interest in having an indication of their mental ability. 

4. Analysis of shortages in the lives of growp members 
Most programs for adult education are not based solely upon the 
expressed interest of community groups, as is possible in the unusual 
organization of the “planned community” of Radburn, New Jersey. 
Leaders usually sense certain deficiencies or inadequacies in the lives 
of their students and seek to fill them. On the basis of known de- 
ficiencies, they frequently set out to guide their group in directions 
beyond the present vision of the students. Adult educators enter 
communities which have no apparent interest in art and lead mem- 
bers to demand concerts or an art gallery. Or, they establish homes 
in the center of a delinquency area and inspire an interest in whole- 
some recreation. Almost every educator can furnish instances of his 
realization of shortages in the habits, the attitudes, or the interests of 
his community and can tell how he is striving to lessen them. 

Since adult educators so often recognize shortages in the lives of 
their students and educate in view of them, the shortages should be 
determined carefully and should be stated. 

Example- 

A method of determining shortages and of founding an educational program 
on them is exemplified in a thesis [13] in which a program is recommended 
for cotton mill workers. A picture of the community as it is today is clearly 
drawn. Ideals of what such a community might become are discovered from 
the writings and talk of regional “frontier thinkers.” The difference between 
the existing picture and the possible one as stated by authorities is subtracted 
and labeled shortages. Then an adult education program is recommended, 
in which ways are shown whereby the shortages may be lessened. 

C. Selection of Study Materials 

Wise selection of curriculum materials will be determined by the 
goals which the leader and the group have formulated; by knowledge 
of the interests, abilities, and needs of the group; and by the wisdom 
incorporated in applying the discovered principles to the preparation 
or selection of materials. 

I . Methods of writing materials 

Many leaders or organizations prepare their own curriculum ma- 
terials. Careful procedures shoulci be evolved for securing adequate 
manuscripts. The adult educators who responded to requests for 
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information used the following methods for preparing curriculum 
material: 

Examples: 

Materials were prepared by experts, in the light of objectives set up by the 
organization and of a general knowledge of the character of its membership. 

Materials were written by professional authors who obtained criticisms from 
experienced leaders. 

A book was written cooperatively by group leaders in conference after a 
trial of tentative outlines with study groups. 

A book resulted from the discussions of a group led by an author who had 
a stenographer taking notes. 

2. Methods of selecting printed materials 
A considerable amount of printed matter which contains material 
fpr courses of study is on the market. The material must be selected 
by people who are familiar with the groups for whom it is intended. 
Although the task of selection is generally delegated to a leader or 
to a committee, an improved method is actual trial of the material 
with all or part of the group who will use it. No methods for basing 
selection on group cooperation have been described in the adult 
education literature. Two examples of possible utilization of group 
aid in selecting its reading are drawn from the present study. 
Examples: 

A committee from the study group may be asked to evaluate potential 
curriculum material. This evaluation should be made in terms of a guide 
sheet. (See Chap. Ill, Table II.) Included in their evaluation should be 
a statement of each book’s interest to them. The leader or the committee 
can then use the analyses to pick the best book for the group to study. 

If the whole group is asked to criticize the material that it studies, standards 
for future book selection may be derived from its criticisms. These standards 
may be obtained in the following five steps. 

1. List the books in the order of their popularity. (See Chap. IV, pp. Syf.) 

2. Study the specific qualities of the books, such as, presentation of figures, 
type of explanations, difficulty. (See Chap. V, pp. 115-26.) 

3. Study comments of the readers, and check their opinions of books against 
the objective analysis of the book. (See Chap. V, pp. 132-37.) 

4. Examine the background of different readers. (See Chap. IV, pp. 84-101, 
and Chap. V, pp. 1 37-41.) 

5. Combine the information about books, readers, and their tastes into general 
patterns or guides which may be followed in later selection. (See Chap. 
V, p. 137, and Chap. VI.) 
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D. Evaluation 

Directors should make periodic check-ups o£ the success of their 
programs in general and in specific phases, in order to obtain the satis- 
faction of seeing progress and to improve upon methods. Among the 
indications of the success of a whole program are, first, measurable 
changes in the lives of the members j second, results of questionnaires 
or check lists 5 and third, comparable records of projects. The suc- 
cess of the specific phases dealing with curriculum materials may be 
discovered by giving well-conceived tests, as well as from the opinions 
of the students and of their leader. 

I. Evaluation of frogram 

(a) Discovering changes in habits of a community or 
group. The habits of a community which might be altered by 
adult education may be studied from a number of angles. Each or- 
ganization may furnish information which will reflect such things 
as selective movie attendance, increased library patronage, changed 
content of newspapers, increased civic interest as expressed through 
concrete enterprises, changed leisure time habits, etc. 

Example; 

The Brooklyn Institute [ii] collected information about differences be- 
tween its students and adults who had not taken courses offered by the Insti- 
tute. A number of students and non-students of vocational courses were 
interviewed. The answers of the two groups to the various questions were 
contrasted in order to find out whether studying vocational courses had an 
effect upon regularity of employment, amount of eainings, and attitude 
toward work. 

Another group of students was questioned in an interview which was fol- 
lowed by a questionnaire regarding their leisure time activities. The replies 
were analyzed by separating the data according to different levels of formal 
education and different courses taken. It was discovered that those who had 
taken cultural couises were distinctly superior in all educational levels in their 
use of leisure time. 

From among a long list of leisure time pursuits and habits, it was found 
that the habits of the vocational students excelled those of the non-vocational 
students only in the amount and kind of reading done in their spare time. 

(b) Use of interviews, diaries, check lists. If interviews, 
diaries, and check lists are carefully prepared and are administered 
in a manner which inspires cooperation, they will reflect any major 
changes that have taken place in the lives of group participants. The 
results should be validated by comparing them with similar records 
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gathered before the members were influenced by the group and by 
parallel records of non-members. 

Example : 

The New York Young Women’s Christian Association conducted a study for 
the purpose of finding out whether attendance at camp had any effect upon 
the leisure time interests and activities of girls. [i6, pp. 7-9.] 

The techniques used to discover leisure time interests were: (quoted ver- 

1. Basic background data serving to define the groups under consideration 
and to facilitate control of important factors, i.e., age, wage, occupation, 
education, nationality, religion, community. 

2. Check list of interests and activities in. terms of degree of interest, partici- 
pation, and limitation. 

3. Check list of social and economic problems, examined for reading, discus- 
sion, and study group membeiship. 

4. Spontaneous reports of reading — boob, magazines, newspapers. 

5. Diary records in half-hour units for three full days — ^work day, Saturday, 
and Sunday. 

A questionnaire and diary were collected from the same girls a second year, 
thus demonstrating the general stability of the data and showing occasional 
shifts in interest. 

(c) Records of projects. All leaders should keep records of 
the development of the program. Accurate reports should be made 
of significant behavior or remarks of group members which reflect 
their criticisms of techniques and their development. These records 
should include accounts of obstacles and methods used in overcoming 
them. They constitute a biography of the venture and give the best 
picture of the group’s accomplishments. They are valuable, also, in 
leadership training. 

Examples; 

The Young Men’s Christian Association prepaied forms obtaining and 
preserving opinions upon the value of group programs. Forty criteria for 
evaluating group activities are listed and illustrated. The leader is required 
to report the proportion of his group which fulfills the ciiterion satisfactorily. 
Examples of the cifiteiia are: “How many tenths of the group have their 
activities characterized by single-mindedness'”’ “How many tenths of the 
group evaluate experiences?” “How many tenths help introduce social changes 
without introducing disorder?” [8, pp. 1-26] 

Dr. Jessie A. Charters teaches parents to help children through a recorded 
project method. When a project is carried on> the parent is required to 
answer these questions. 
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What is the difficulty which you will teach the child to overcome? How 

do you think the difficulty arose? Is the child’s interest aroused? Write 

down what you tell the child about learning this act. Describe everything 

that happens as he is learning. [2] 

2 . Evdmtion of materials 

The success of the curriculum materials employed m the program 
may be discovered from tests, from the opinions of the students, and 
from the judgment of their leader. 

(a) Testing the effect of materials on students. One 
way of discovering the clarity and the efiFectiveness of materials is the 
administration of tests. One type of examination tests the amount 
of information that students have acquired. The leader may com- 
pose his own factual test, using a simple objective form. If the test 
is given before the discussion of the textbook, the students’ responses 
to a good test will often show deficiencies in their study materials. 

Another type of test measures changes in attitudes. Attitude tests 
and scales may be used at the beginning and at the end of courses, 
to demonstrate to both students and teacher what changes have been 
wrought in their viewpoints. Many tests upon problems which are 
commonly discussed by adult groups have been published. A few 
of these are: [14] 

Character and Personality Test Series. Opinions on International Questions. 
Association Press. 

. Qftnions on Race Relations. Association Press. 

Katz, D. Attitude Toward the Law. University of Chicago Press. 

Peterson, R. C. Attitude Toward Cafital Punishment. University of Chi- 
cago Press. 

. Attitude Toward the Chinese. University of Chicago Press. 

Thurstone, L. L. Attitude Toward Communism. University of Chicago 
Press. 

. Attitude Toward Honesty in Public Office. University of Chicago 

Press. 

(b) Utilizing the leader’s opinion. The leader’s evalua- 
tion of the material which his study group used should always be 
obtained. First, each leader should keep a record of his success with the 
material: the informal comments of readers and the specific points 
where he encountered difficulty in teaching should be written. Sec- 
ond, the curriculum director should ask all leaders to respond to 
questions prepared by him regarding the interest and apparent com- 
prehension of the class as it used the materials, the effectiveness of 
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different sections, the comparative appeal of different styles of pres- 
entation, etc. 

(c) Utilizing the study group’s opinion. The general 
value of readers’ opinions has been presented before. The use of 
part of the last meeting in a course as a “testimonial meeting” has 
been found a satisfactory means of obtaining these opinions. Reader 
annotation of library cards has, also, been used as an acceptable de- 
vice for getting the group’s opinion. 

Example: 

The last method was the basis of the Winnetka research with children for 
grade placement of interest in certain books. [17] Children were asked to 
write the name of author and title on one side of a card and to give their 
opinion of the book on the other side. They were asked to make a statement 
that would tell other children whether or not they would like the book. This 
method has been adapted by Elizabeth Morriss in her compilation of a book 
list for adults of limited reading ability. [15] 

SUMMARY 

The curriculum in adult education is defined as “a planned pro- 
gram of study, set up with methods and materials indicated for the 
fulfillment of definite aims.” This definition emphasizes the value 
of planning a program, the need of properly selected materials, and 
the necessity of determining whether the materials and methods 
actually fulfill the original plans. 

The aims of a study group must be formulated primarily on the 
basis of its own interests and needs. The leader and the group must 
decide whether reading material is a desirable instrument, they will 
find that reading is generally an important means of attaining their 
ends. The group must decide upon the other activities besides read- 
ing that should be introduced. 

Although formulating the aims and deciding about the use of 
certain methods and materials is a creative task that rests largely 
with the gi'oup, practical suggestions from the exp,erience of others 
may be offered m the selection of the best materials to carry out 
purposes. A survey in 1934 showed that a sampling of adult edu- 
cators had devised a number of methods for finding the best cur- 
riculum materials for their courses. Some of them asked authors 
to write material to meet certain requirements; others asked group 
leaders to evolve study materials, using their groups as a laboratory for 
testing the lessons; still others worked out methods of discovering the 
printed material which would probably fit the needs of their groups. 
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The survey showed, too, that educators had formulated some meth- 
ods of evaluating the success of their programs. These evaluations 
were usually directed at finding important changes in attitudes and 
habits which participation in an adult education program might 
induce. They were almost never directed toward a criticism of a 
specific device, such as the course of study. 

Methods of obtaining and evaluating a curriculum may be recom- 
mended. Many of these methods can' be illustrated by present prac- 
tice in the field of adult education, but one phase of the process, 
obtaining a group’s aid in selecting and evaluating reading matter, 
has no counterpart in practice and must be illustrated from the pres- 
ent study. The recommendations are in four steps. 

A. The plans for an adult education organization should be formu- 
lated by leaders and representatives of the group. The plans should 
be based on the previous year’s experience, the general purposes of 
the organization, and the purposes and interests of new groups. 

B. The group for whom the program is planned should be studied 
in order to find its interest, abilities, and needs. A questionnaire or 
a more formal survey will reveal basic information about group mem- 
bers. Their answers to questions will show the direction of their 
interests. Tests or informal questions may be administered to ascer- 
tain some of the abilities of the members. All of the above sources 
of information will indicate inadequacies in the past experience or 
outlook of the members, inadequacies that should be carefully deter- 
mined and then be frankly faced by leaders. 

C. The plans of the group and information about the group will 
show the areas in which education should take place. If reading 
matter is to be used, it should be carefully selected. Course of 
study material may be written by experts who know the type of 
membership that will use it, by group leaders who try out the ma- 
terial with groups, or by the group itself, from stenographic records 
of its discussion. 

If printed material is to be selected, the members of the group 
should aid in choosing the material that is most interesting to them. 
Selection will be more satisfactory if it is based upon the group’s 
criticism of previous books. 

D. There are several ways of evaluating an adult education pro- 
gram. Statistics about the recreational and other habits of a group 
for the period before participation in the program may be compared 
with the same sort of statistics collected after participation. The fig- 
ures on attendance, circulation of books, etc., may be obtained from 
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such agencies as the library, Sunday school departments, or moving 
picture box offices, or they may be obtained directly from interviews, 
questionnaires, diaries, or check lists written by group members. The 
leader, too, can furnish valuable records in a diary of his activities in 
relation to the project. 

The course of study may be evaluated by asking the students to 
submit to tests which might show the elfect of the material on their 
attitudes and its contribution to their information. The records of 
the leader may again give a satisfactory picture of the advantages 
and' shortcomings of the material. Finally, the students can play a 
major part in evaluation by making a careful criticism of the material 
that they have studied. 
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CHAPTER HI 


AIDS IN SELECTING AN ADULT EDUCATION 
CURRICULUM 

In Chapter II methods which have already been worked out for 
studying an adult group were stated. It was shown that some tech- 
niques for evaluating a total adult education program have been 
formulated. In the field of the curriculum, several ways of obtain- 
ing reading materials have been evolved. Little help could be ob- 
tained from practice, however, at the crucial points of selecting that 
reading which is best fitted to individual groups and of evaluating 
the strength and weaknesses of the particular material chosen. 

Experienced educators, of course, have not been sitting idle, wait- 
ing for technicians to tell them what people want to read. The per- 
son in charge of planning and preparing program material for a na- 
tional organization usually possesses a broad understanding of people’s 
preferences which gives him a clue to reading selection. This experi- 
ence needs to be pooled. It requires validation by experimentation. 
It also demands formulation in general rules which may be applied 
to material written for specific groups. 

Insofar as the persons who are preparing bibliographies or litera- 
ture for groups have been analyzing their material and have been 
studying their groups’ abilities, interests, and needs, they have been 
following the principles enunciated in this study. It is urged that 
the program chairmen take one further important step. On the 
basis of the analysis of material and of groups, leaders should state 
in writing the qualities of specific books that appealed to particular 
sorts of readers. The equally important antithesis should be stated: 
namely, which factors caused lack of success of some materials. Such 
written records would be most valuable if they were contributed to 
a central clearing house by many workers. This center of investi- 
gation would synthesize all contributions into general rules of selec- 
tion of the best reading material for many types of groups. 

The methods which have been worked out in this experiment for 
studying group members’ opinions of the reading material and 
forming generalizations therefrom will be discussed in this chapter. 

34 
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Adaptations of these methods to other investigations will be suggested. 
Problems for further research will also be presented. 

METHODS OF STUDYING READING MATERIAL 

A leader who follows the procedures suggested in Chapter II for 
determining the curriculum materials for his adult study group will 
first establish a goal for the program. The group would preferably 
cooperate in formulating the goal and in selecting the subject and 
the topic for study. The leader will then probably assemble as 
many printed treatments of the topic as possible and find the one that 
would be most interesting to the group that he will teach. 

In order to determine the best materials, it is necessary for the 
leader to possess three bodies of information: 

I. The qualities that make any book interesting or uninteresting. 

2,. The amount or degree of each quality possessed by the various 
books that he is considering. 

3. The amount or degree of each quality that this particular group 
demands in order to be persuaded that a book is interesting. With 
this information at hand, the leader can generally select the book 
that comes closest to fulfilling the demand of the group for inter- 
esting reading. 

The present experimental study assembled a list of the qualities 
that made certain books interesting, discovered the degree of some 
of these qualities in the books, and estimated the experimental read- 
ers’ attitudes toward the selected qualities. After the problem has 
been defined more clearly, these three processes will be described. 

The intricacies of fulfilling the requirements of setting up stand- 
ards may be pictured in the following problem, a problem stated in a 
form simpler than the actual process of erecting standards. 

A definition by illustrMion. Here in my hand is a book of poetry 
that was written by a group of fathers and mothers in our Authors’ 
Club. You and I have been reading snatches, admiring the swing 
of some of it and the originality of other bits. The telephone rings. 
A city editor of the newspaper has heard about this book of poetry 
and wishes us to send him a copy of the poem from the book that 
his readers would most enjoy. 

At first this looks like a simple task. I have my favorite poem, and 
you have yours. I say, however, that your favorite jingles un- 
pleasantly; you claim that there is nothing more trite than the third 
line of mine. In addition, we realize that neither of these poems 
might appeal to the newspaper readers. We find ourselves at an 
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impasse, where our feelings, plus the unorganized ideas that we 
have, are not sulEcient to make a trustworthy decision. 

Under normal circumstances we would spend another half hour 
in spirited debate and would finally compromise on the least objec- 
tionable, and probably the most colorless, of the choices. Today, 
with time to spare, you and I decide to be more thorough in our 
selection, to try to answer squarely the editor’s challenging request 
that the poem be one which would best represent our group of 
parents to the public. 

After some trial and error, it becomes evident that the selection 
of a poem presents at least three different problems. We began 
by trying to discuss them all at once. We now see that we must first 
decide upon some of the qualities of poetry. Then we must find 
standards which will tell us what the range of desirability in these 
qualities is. Finally, we need to know which standards the readers of 
the poetry will apply when they judge its qualities. 

Stef I. Turning through the book, we jot down at random a few 
qualities of poems; rhythm, subject matter, use of words, figures of 
speech, and rhyme. These will furnish a starting point and will 
show the kinds of qualities we must keep in mind in deciding which 
poem is best suited to the readers. Such a list in itself is helpful, 
for it enables us to think through the problem, rather than to attack 
it merely by intuition. 

Stef 2. Setting down the names of qualities is no more than a 
beginning. It is necessary to examine the various poems in the light 
of each element and to ask: What is good rhythm.? Which subjects 
are appropriate for a poem and which are hackneyed.? Are some 
choices of words better than others.? How will we tell a remarkable 
combination of words from one that is just better than average ? When 
are figures of speech acceptable, when undesirable? May poems 
which have no rhyme be considered real poetry? 

In other words, the list of qualities of poetry directed our atten- 
tion to the things that the readers will judge. They will apply their 
standards as they evaluate each quality. Our second job is to find how 
these qualities vary. For instance, in respect to the quality of rhythm, 
we may ask, what is the poorest rhythm that is accepted by any judge, 
and what is the best that the readers would recognize.? Our answer 
to that type of question can come from several different sources, of 
which we select two this afternoon, one based on our own preference 
and the other on expert advice. We place the poems in a row on 
the table, in an order from one with the poorest rhythm to the poem 
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with the best, according to our taste. In the same way we place the 
poems in order for each of the other four qualities. Before we can 
rank the poems upon the quality of their subject matter, we ask for 
help from an English teacher who is acquainted with the opinions 
of many criticis in regard to the best and the least desirable subject 
matter for poetry. His expert advice guides the ranking of the 
poems on that quality. 

Stef 5. The long afternoon is almost over, and we are only now 
ready to select the best poem, after we have evaluated the quality 
of all of them. The task should be simple, you sayj why not choose 
the poem that we considered to rank highest upon the most quali- 
ties'' If our purpose is to select the poem that the newspaper readers 
would enjoy, we do not know that they would concur in our judg- 
ment. A new group of questions arises. Which degree of those 
five qualities do the readers judge to be best? Do they like the verse 
that ranked medium in our rating of rhythm; might they be most 
amused at the rhythm which we ranked lowest; or might they prefer 
the rhythm that we placed highest? What standards do they apply 
in judging the qualities of poetry? 

In order to answer these questions, we think of the tastes of our 
friends. But even if we know them, are their tastes typical of the 
average townsman? We telephone the editor for help. “Yes,” 
he says, “I have always wondered about that, too.” We table the 
question with a promise that when we are in a scientific mood again 
we will try to discover what kinds of poetry the community prefers. 
We send the editor four poems, ranking at different points on the 
scales of quality, which we think would appeal to the tastes of four 
different types of newspaper readers. 

Discovery of the Stmdctrds That Readers Use in Evaluating Ma^ 

terial 

Before we could discuss the selection of poems, it was found neces- 
sary to state the various qualities upon which poems should be 
judged. Only five elements were listed for the simple task of se- 
lecting a poem. When the selection and evaluation of large units 
of work depend upon the adequacy of topics which form the basis 
for analysis, the need for as complete a list of topics as possible is 
obvious. In accordance with this belief, the present study was 
chiefly concerned with the determination, definition, and classifica- 
tion of topics upon which criteria for judging literature are based.^ 

'‘See Chap. IV, pp. 71-83. 
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When interviews and books on adult reading were consulted, all de- 
vices for estimating the value of general non-fiction literature were 
noted. The experimental study was based, however, upon an anal- 
ysis of books about current controversial problems. This field was 
the focus of attention for several reasons. First, there is a widespread 
interest on the part of adult groups in current problems. Second, 
generalizations from introductory treatments of current problems 
might be more readily applied to other fields than generalizations 
from a study of the more specialized writings, for example, of parent 
education or agricultural extension. Third, less has been discovered 
in the field of social education than in other branches which have 
drawn many learners for the past quarter of a century. 

Four different sources of criteria were examined. One was in the 
field of opinion, the data for which were collected from interviews 
and reading. Three were derived from an experiment in which 
readers judged pamphlets before reading them, then made general 
criticisms of the pamphlets after reading them, and finally gave de- 
tailed criticisms of sections of the pamphlets. 

The writings of adult educators, librarians, and publishers were 
combed for suggestions. Many people who prepare or select manu- 
scripts were interviewed. The survey of literature and the interviews 
showed many deficiencies in knowledge or in adequate analyses. It 
became evident that first-hand information must be obtained. Through 
the service of Columbia University, a group of college people em- 
ployed in the Civil Works Administration was assigned to the ex- 
periment. 

The Civil Works Administration group of forty-five workers read 
twelve pamphlets upon current problems: problems of war and peace, 
unemployment, changing government, and unstable money. Each 
of these four subjects was discussed in a series of three pamphlets. 
The task of the readers, as explained in the mimeographed instruc- 
tions, was: “To discover your interest in various pamphlets, your per- 
sonal reaction to them. You will be asked to rate them. But your 
own feeling, which you are called upon to state in answer to ‘Why’ 
or ‘Reason,’ is even more important than your rating.”^ Instructions 
covered uniformities in procedure and permitted honest emotional 
and intellectual reactions. 

The influence of subject matter upon the interest in particular 
pamphlets was recognized and was accounted for as far as possible. 
Before any of the pamphlets were circulated, the readers were told 

“ See Appendix II A. 
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to follow two directions in regard to the four subjects dealt with by 
the pamphlets. First, they checked any of the topics that they would 
ordinarily be interested m reading about. Second, they ranked the 
topics in the order of interest to them. Only three out of forty-five 
readers were interested m no topics, and eleven were interested in all 
five. After they had finished reading the pamphlets, the readers 
were again asked to rank the topics, this time in the order in which 
the pamphlets had interested them. (See Table X.) The average 
rankings before and after reading were similar. The topic of un- 
stable money was consistently unpopular. The topics of unemploy- 
ment and of war and peace exchanged first ranks but were both popu- 
lar. In all interpretations of results the influence of the subject as 
indicated in the interest rankings should be recalled. 

Three different methods were used for eliciting the reasons for 
interest in each pamphlet. First, after studying a booklet, the reader 
checked each section® on a five-point scale, from “very interesting” 
to “dull.” As he made each check upon the scale, the reader was 
asked to record why he had made the rating. These sectional reac- 
tions showed in detail the standards used in judging material. Then 
a general criticism in the form of a brief report concerning total im- 
pressions of the pamphlets was requested. This uncovered points 
which might have been overlooked in the section reactions and rein- 
forced some of the opinions expressed in the detailed analysis. Fi- 
nally, since three pamphlets upon the same topic were distributed at 
once, each individual recorded what is termed his “snap judgment” 
preference. Here he stated which pamphlet appeared so interesting 
that he wanted to read it first. 

With the aid of three workers, the data from the readers’ criticisms 
were assembled. The different reasons for liking or disliking the 
pamphlets were recorded verbatim. The large number derived from 
section reactions were written on separate cards for easier classifica- 
tion. The general criticisms and the snap judgment preference of 
each pamphlet were catalogued in some detail on separate sheets. This 
presented succinct pictures of the opinions about each book. 

The author unified the classification of the reasons, which had been 
collected by three assistants. Thirty-eight standards dealing with 
different aspects of judgment were distinguished. As the standards 
are now expressed in brief topical headings, they are no more than 
names of qualities of books. The readers’ positive and negative 
criticisms which were classified under each heading interpret them 

“ Each pamphlet was divided into from eight to twenty-nine sections. 
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as standards to be applied to the evaluation of qualities. The topics 
do not become real standards until they are defined by the readers’ 
comments. The different comments are given in the classification un- 
der each standard in Chapter VI. 

In the brief survey of the methods of discovering topics for criteria, 
four different sources were used. Each source suggested or empha- 
sized topics partially or completely ignored by others. The first 
source of topics for standards was the opinions of writers and workers 
in adult education. They named most often (all lists are ranked in 
order of frequency of mention) : 

1. Techniques for getting and holding reader interest. 

2. Difficulty of concepts. 

3. Dramatic style. 

4. Size of the book (its brevity, in particular). 

However, as the experimental readers commented upon the different 
sections of pamphlets, they mentioned most often; 

1. Clarity of explanations. 

2. Selection of material. 

3. Techniques for getting and holding reader interest, 

4. Methods used in introduction for awakening interest. 

In the general criticisms, the experimental readers’ emphases were 
somewhat different. They commented most frequently upon: 

1. Difficulty of concepts, 

2. Quality of information presented. 

3. Vitality of the style. 

4. Emotional appeal. 

Their reasons for selecting one pamphlet to read first on a snap judg- 
ment impression were primarily: 

1. Title. 

2. Previous familiarity with author. 

3. Selection of material in pamphlet. 

4. Format — cover, illustrations. 

The criteria from four sources were combined in a single classified 
list. The Combined List of Criteria, stated in full in Chapter VI, 
includes all the reasons given in the opinion list, the section reac- 
tions, the general criticisms, and the snap judgment summary. Each 
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of the 4,524^ reasons found in this study fits under one or another 
of the topical headings. 

The type of response and the way in which it was classified may 
be demonstrated briefly. The following sample of reader responses 
is taken from the Combined List of Criteria: 

Positive Comments Negative Comments 

A. Content 

B. Form 

I. Difficulty 


2. Literary quality 

a. Organization of material 

(1) Shows unity (i) Disjointed; incoherent 

(2) Direct; to the point (2) Main topic sidetracked 

(3) Systematic arrangement; (3) Lacks organization; uninteresting 

good correlation arrangement 

b. Form and grammar 

Simple statement that outline (or other) form is piesent 

(1) Interesting form (i) Too much outline (or other) form 

(2) Sentence structure good (2) Sentences poor; too long 

(3) Poor grammar 

c. Vitality 

(1) Well worded statement, (i) Poor style; badly written 
good style 

(2) Spicy; colorful; racy; (2) Shoddy, formal; dead; dry; 

conversational stereotyped 

The readers were quite conscious of the organization of the 
pamphlets and made many comments of the type given here. They 
were less interested in the form and grammar. Vitality was one of 
the standards most frequently mentioned. Under it were assembled 
the adjectives describing qualities which are difficult to analyze, but 
which express those important intangibles that either hold or lose 
readers. 

The detailed list of thirty-eight criteria® was necessary in order to 

‘ Number of reasons upon which the Combined List is based 

(a) Opinion 70 

(b) Section reactions 3,300 

(c) General criticisms 926 

(d) Snap judgment 228 

Total 4,524 

' See Skeleton List, showing thirty-eight criteria and frequency ranks. Chap, IV, 
Table V, pp. 82-83. 
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analyze the data of the experiment and to discover all aspects of the 
reasons given for liking or disliking the pamphlets. The main topics 
of that list follow: 

A. Content 

1. Historical allusions 

2. Presentation of figures 

3 . Selection and quality of figures 

B. Form 

1. Difficulty 

2. Literary quality 

3. Literary techniques 

4. Format 

C. Catching and holding reader interest 

1. Introduction 

2. Conclusion 

3. Techniques for relating problem to reader 

4. Emotional appeal 

5. Use of famous author’s name 

D. Dealing with controversial issues 

1. Author’s display of his own viewpoint 

2. Techniques of controversy 

After all of the research evidence concerning the importance of the 
various standards was accumulated, an abbreviated list was devised 
which should serve the practical purpose of aiding others in the 
selection of literature in at least the current problem area. The 
originab Combined List of Criteria was rearranged under functional 
main topics. Thirteen of the fourteen main criteria in the old form 
were retained with rewordings. Because of its infrequent mention, 
the topic use of famous author., indicating the influence of a distin- 
guished name upon the reader, was omitted, and literary techniques 
and literary quality were divided into four rather than two topics. 

This list is presented in a form which could be used with groups 
without much modification. Preliminary try-out has indicated it to 
be self-explanatory. Groups with a limited vocabulary might require 
simplification of the more difficult words, such as graphs, data, clarity, 
and viewpoint. With such changes taken into account, the classifica- 
tion should produce a well rounded appraisal of controversial litera- 
ture. The new list of standards is worded as though a magazine 
article were to be judged. A few changes in terminology should per- 
mit the list to be used for pamphlets or books quite as easily as for 
articles. The list is shown in Table 11. 
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Title of article Your name 

In your estimation, what are the chief values and weaknesses of this 
article? Consider the article carefully in regard to each of the follow- 
ing topics* and state your opinion of it beside each topic. 

TOPIC COMMENT 

Interesting the reader 

Title and appearance' of the article 

Clarity and appeal of introduction^ 

Methods of making article seem important’ 

Difficulty^ of the material 

Literary quality® 

Clarity and interest of material’s organiza- 
tion® I 

Informing the reader 

Quality of explanation’ 

Use of examples, stories, comparison® 

Choice and quality of information® — - — 

Presentation of figures, tables, graphs'® 

Introduction of data from history" — 

Helping the reader to decide 

Statement of author’s viewpoint" 

Methods of presenting arguments and criti- 
cisms" — 

Appeal to emotions" — 

Summary and inspiration at conclusion" 

Was this article unusually interesting; rather interesting; 

dull? State your generd opinion of the article briefly on the 

back of this sheet. 


* Information about each topic is given in Chapter VI at the indicated page. This list 
should not be used until the references are studied. 


^ Format, p. i88. 
i Introduction, p. igo. 

I Catching reader’s interest, p. 193. 

* Difficulty, p. lyi. 

'Vitality, rhythm, simplicity, adult ap- 
proach, pp. lypff. 

® Form, organization, p. 178. 
i Expository techniques, p. 182. 


* Dramatic, figures of speech, p. 181. 

• Subject treated, p 170. 

Figures, p. 167, 

Historical allusions, p. 1 66. 

"Author’s viewpoint, p. 196. 

" Controversial techniques, p. 198. 

" Appeal to emotions, p. 194. 

“ Conclusions, p. 191. 



44 BOOKS FOR ADULT STUDY GROUPS 

Before it is possible to utilize the responses to the guide, the full 
discussion of each topic in Chapter VI must be consulted. This treat- 
ment of criteria gives the complete classification of each topic with 
detailed information about it. 

Analysis of Qualities of Standards by Examining Pam-phlets 

The listing and interpretation of the criteria® which lead readers 
to accept or reject material, is a new step in the selection of curricula. 
Never before to our knowledge has a group of adult readers sys- 
tematically recorded its impressions of materials in terms that could 
be reduced to standards. A qualitative picture of each standard is 
presented by the simple device of classifying all the shades of objec- 
tion to or approval of the pamphlets under the appropriate heads. 
For instance, under “Organization of material” (page 41) it may 
be noted that readers like pamphlets that are unified, direct, and well 
orderedj they dislike materials that sound disjointed, that are poorly 
organized, or that digress. 

It is not enough to list criteria with descriptive adjectives. The 
complete picture demands knowledge of the literature to which the 
adjective refers. One should ask: Which material is disorganized 
and which is judged to be unified? What makes it so? If the ma- 
terials that possess dilferent degrees of each standard are examined, 
the qualities of each criterion will become tangible. This examina- 
tion was performed in the present study by obtaining qualitative 
rankings from readers and by an objective analysis of treatment of 
standards in each pamphlet. 

Reader analysis of concrete material is one instrument in answer- 
ing the above questions concerning the nature of criteria. Ordinarily 
critics should compare several books in the process of analysis. When 
an average reader evaluates a single book he is notoriously apt to 
eulogize it, or at least to make so many favorable comments that his 
response is valueless. The experimental readers accordingly were 
asked to compare three pamphlets with one another, ranking them 
numerically in order of desirability. In order to rank them, it was 
necessary for the readers to weigh the qualitative factors against one 
another and to distinguish degrees of interest or of quality in three 
related books. 

Although thirty-eight standards or qualities were discovered, they 
could not all be analyzed in this study j so a few important qualities 
were singled out to be observed carefully. The experimental group 

' See p. 41 and Chap. VI. 
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was asked to rate each series of three pamphlets upon six qualities. 
They ranked each pamphlet i, 2, or 3 upon attractive appearance, 
interest of figures, quality of writing, ease of xmderstanding, appli- 
cation of explanation to personal needs, and relation of subject to 
personal problems. 

The group agreed closely upon the order of ranking for the appear- 
ance of the pamphlets and agreed to a less extent upon rankings for 
difficulty and for quality of statistical presentation. The group’s 
rankings of quality of writing, application of explanation to personal 
needs, and relation of subject to personal problems were little more 
than random. 

As a result of the rankings of the pamphlets upon the quality of 
appearance, that standard can now be placed upon a scale of reader 
preference. We know that the readers rated as unattractive two pam- 
phlets with small, poorly arranged type, and one with unusually 
poor paper. Two of the least liked had cartoons on colored covers. 
They rated all the pamphlets published by a particular house, pam- 
phlets with heavy paper and twelve point type interrupted by bold 
face section headings, as average in appearance. Among their first 
choices two of the three were printed on glossy paper, with a style 
of type and page that resembled books. The third first choice had 
large type, several attractive graphs, and a picture. As a result 
of reader ratings on the topic, “attractiveness of type and page,” a 
scale of qualities was obtained in which glossy paper, average sized 
type, and illustrations were higher in desirability than cheap paper, 
small type, and flashy covers. 

The readers’ rankings of the pamphlets in their order of difficulty 
were different from the formula of difficulty'^ which was applied ob- 
jectively to the pamphlets. The divergence in viewpoint may quite 
as plausibly mean that the formula is insufficient as that the readers 
were undiscerning. 

Useful as the opinions of the readers were in indicating some of 
the qualities upon which book selection is based, another sort of ap- 
proach to the problem was needed. The same qualities of the ma- 
terials were examined objectively in the laboratory. This provided 
a check upon readers’ judgments and was used to probe into reasons 
for judgment of which the readers were not fully conscious. 

The present study experimented with several different ways of 
determining the effectiveness of reading matter. There are three 
illustrations of the quantitative approach to analyzing the nature of 

’See p. 46 and Chap. V, p. iji. 
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booksj one of a qualitative approach, and three of descriptive meth- 
ods. The twelve pamphlets were subjected to analysis of quantity 
of history, quantity and kinds of figures, vocabulary, quality of style, 
description of format, techniques of catching and holding reader inter- 
est, and author’s presentation of controversial issues. 

The amount of a quality in the pamphlets was compared with 
reader preferences to find which amounts were most popular. The 
general technique was a correlation of, for instance, historical allusions 
with a preference rating of the pamphlets. A high positive correla- 
tion would show that the larger the amount of history, the more 
popular the pamphlets were apt to be. The instrument used to indi- 
cate preference was the Pamphlet Preference Index, compiled from 
two different ratings of pamphlets.® In the Index the two most 
popular pamphlets were tied with a rank of 1.5; the least popular 
pamphlet was ranked 12. Difficulty was determined by finding the 
proportion of easy and hard words in a sampling of one thousand. 
The Thorndike Word Book [3]* provided rules for determining 
difficulty. In order to find the ratio of simple to complex words, 
a sampling of seventy-five sentences was obtained. The two measures 
of proportion of words and of sentences were set into the Winnetka 
formula [4], which indicated the grade placement of each pamphlet. 
The twelve pamphlets were ranked according to grade placement in 
an order called the Difficulty Index. It was possible, within the 
limits of the data at hand, to compare preferences for pamphlets with 
their ranking from hardest to easiest. Although the rank order cor- 
relation® was low,^® the pamphlets that were average in difficulty were 
more popular than either the hard or the easy ones. 

Next, a scale for the proportion of allusions to history among the 
total number of pages in pamphlets was established. In each pamphlet 
the number of lines with historical allusions was counted. (History 
was defined as “Any reference to data prior to Roosevelt’s adminis- 
tration.”) The pamphlets were ranked in order from the greatest 
amount of historical information to the least. The correlation’-^ be- 
tween preference for pamphlets and quantity of history indicated that 

* Throughout this chapter numbers in brackets refer to bibliographical references 
to Chapter III, as given on page 68. 

® See Chap. IV, pp. 87 f. 

“ The rank order method of correlation (symbol is rho) is used throughout. 

“Rho for difficulty -was -b.o9. FuU description in Chap. V, pp. 116, 134, and 
Chap. VI, pp. 172-77. 

“■Rho was — .46. For a full description of methods and findings see Chap. V, 
pp. iig, 134, and Chap, VI, p. 166. 
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readers tended to prefer those pamphlets with comparatively small 
numbers of historical allusions. A more detailed, investigation is 
necessary to discover how history may be most effectively introduced 
into reading material. 

The same procedure as that described in the preceding paragraph 
was followed for a study of the numerical information. The number 
of pages containing figures or quantitative information was ascertained j 
and pamphlets were ranked in order from greatest to smallest pro- 
portion of pages with figures. The pamphlets with fewer figures 
were definitely preferred to those possessing more figures,^^ There 
was another refinement in studying the effect of figures on preference. 
The preference ratings of aU the pamphlet sections containing figures 
were contrasted with the ratings of sections without figures. The 
average preference of sections without figures was substantially better 
than the average with figures.^® 

The attempt to judge pamphlets on their literary quality was un- 
successful in this study. The judges of literary merit were a jour- 
nalist, a self-styled ‘‘author,” a librarian, and a modern language 
teacher. They were asked to determine the quality of the writing, 
irrespective of content, in each of the twelve pamphlets. A guide 
list prepared by the judges stated the factors entering into the literary 
merit of expository material: clarity, apt choice of words, proper 
grammar, and refreshing presentation of viewpoints. After consid- 
eration of these points, the three pamphlets on the topic of unem- 
ployment were ranked in order of literary rtierit. Likewise, the 
pamphlets on each of the other topics were ranked in their series 
of three. 

The judges’ agreement in ranfang was little more than random. 
Conversation with each judge indicated so many logical reasons for 
erroi'S in ranking that the submission of pamphlets to others to mul- 
tiply judgments could be presumed to multiply errors. The four 
judges of literary merit were sure that all of the other factors of 
difficulty, content, appeal to personal interest, etc., were so strong that 
they made independent determination of literary merit an impossi- 
bility. This opinion was corroborated by the high disagreement 
among the forty-five experimental readers when they ranked series 
of pamphlets for literary quality. 

“Rho was — .59. For a full description of method and findings see Chap. V, pp. 
iao-12, and Chap. VI, pp, 167-69. 

“See Chap. V, p. 135. 

“See Chap. V, p. 126. 
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The difficulty of determining literary merit by vote indicates the 
obstacles in the path of those who wish to make any other qualitative 
judgments of criteria. It is comparatively easy to prepare scales 
which indicate varying quamities of criteria. But quality and indi- 
vidual values are so closely interwoven that it will not often be pos- 
sible to erect for any given criterion qualitative scales which are un- 
related to specific materials and groups. 

There are many topics for which no quantitative scale can be 
erected, for which qualitative scales are invalid, but upon which it is 
desirable to have as much information as possible. For instance, one 
of the prime factors determining readability of material is the way 
in which the author catches the reader’s interest. The basic method 
of catching interest is to show the material’s relation to the life of 
the reader and to stimulate the reader to broaden his knowledge. It 
is foolish to count the number of times the reader is addressed per- 
sonally, or to attempt to reduce as variable a factor as “human inter- 
est” to a good, bad, or indifferent category. But no study of ma- 
terial would be complete which did not show what methods the 
author used in attempting to reach his reader. 

The simple technique of asking several workers to list the methods 
used by authors in injecting “human interest” into their pamphlets 
was utilized in the study. Since the subject matter is probably the 
single greatest cause of interest, the content of each booklet was de- 
scribed in a paragraph. The assistants then listed “the techniques 
used by the author to catch and to hold reader interest.” No correla- 
tion between preference and techniques of catching reader interest 
could be made. It is interesting to note, however, the techniques 
which were used by the most popular and the least popular authors.^® 

A method of description similar to the above was used in analysing 
the format of pamphlets. Each was described with reference to 
paper, type, cover, pictures, kinds of sections, and size.^“ 

In determining the bias of authors, the descriptive technique was 
condensed into classifications permitting easy comparison. Four 
judges assigned the method of presenting controversial issues in each 
pamphlet to one of four described classifications. The judges were 
more consistent in this than in their rating of literary merit. The 
same classification was assigned to a third of the pamphlets by all 

“See contiast of two pamphlets, Chap. V, pp. 142-55 and List of techniques, 
Chap. VI, p. 193. 

“See Chap. V, p 115 and Chap. VI, pp. 184-89. 
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four judges} they differed by only one classification interval upon 
another third of the pamphletsd''^ 

Several methods for studying criteria in relation to materials have 
been discussed and illustrated. The illustrations are inadequate. A 
multitude of studies of a similar type would be necessary to deter- 
mine all the ways in which material may be presented. Investiga- 
tions of the general nature discussed here may be undertaken by field 
workers in the usual course of selection of curriculum materials. 
More efficient methods are possible in the laboratory than in field 
work. There material is scientifically controlled} the criterion under 
observation is the chief variable. Several examples of such studies are 
presented in Chapter VI under the respective topical headings.^® 
Such experiments are not, however, always as valid as are the more 
lifelike analyses in which typical readers respond in various ways to 
their usual reading matter. The problem of determining the criteria 
that signify readability will best be solved when it is approached from 
the laboratory and from the field simultaneously. 

Analysis of Standards by Insf acting Grouf Criticisms 

Seven qualities of the experimental booklets have just been ex- 
plored as objectively as possible. Certain facts about the group that 
read the booklets should be analyzed. Then the group’s opinions 
about the material with known characteristics should be interpreted 
in light of the knowledge about the group. When data about the 
group, the pamphlets, and the group’s opinions are collected and co- 
ordinated, it will be possible to state with some certainty the standards 
that the group used in judging the pamphlets. 

The first essential in studying a group is the collection of all of 
the information about it which might be relevant to the problem. A 
survey of the literature of experiments upon interests in reading 
suggested that the following factors are probably related to tastes: 
sex, marital status, occupation, age, education, and participation in 
adult education activities. This study sought information about these 
items, but included other items which were related to the problem 
of influences upon attitudes toward controversial issues. These pos- 
sible influences were: religious affiliation, political alignment, educa- 
tion in social sciences, and reading in magazines, newspapers, books, 
and pamphlets. 

""See Chap. V, Table XV, p. iz4. 

“ For examples of experiments under controlled conditions see Chap. VI, pp, r 68, 
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A questionnaire^® containing those items, when filled in by mem- 
bers o£ the experimental group, gave a rough picture of the abilities, 
achievements, and pertinent interests of the group. The questionnaire 
aided in interpreting confusing data. 

One way of using the information about the group may be related 
briefly. It was soon apparent, as the data about the readers’ interest 
in the pamphlets were analyzed, that readers disagreed on their 
preferences. Reasons for tlie disagreement were sought in the nature 
of the readers. Sub-groups which were alike in some respect were 
contrasted with sub-groups opposite to them to see if the two dis- 
agreed more strongly than the group at large. These sub-groups 
were determined by examining the information about each person on 
the questionnaire. 

Several sub-groups were contrasted.®® Among them were groups 
differing in sex and groups differing in education. The ratings for 
pamphlet preference made by all the men were averaged. These 
ratings were compared with the average preference ratings of all the 
women. The men agreed very closely with the women in their 
pamphlet preferences. In the same way, the preferences of a sub- 
group of eight people who had been to college for two years or less 
were compared with the preferences of eight people who possessed 
postgraduate degrees. These two sub-groups disagreed widely upon 
fully half of the twelve pamphlets, approaching agreement upon only 
two of them. The difference in sex did not appear to affect the 
preferences for pamphlets} but there were grounds for supposing that 
variations in education might have produced some of the differences 
of opinion found in the group at large.®^ 

This example shows how differences in preferences for pamphlets 
may be occasioned by the nature of the readers. Differences may also 
be induced by factors within the pamphlets. Two illustrations will 
suffice to show, first, how the} group’s variation in preference for 
pamphlets may aid in finding out what quantity of a particular stand- 
ard they find most interesting, and second, how their preference may 
indicate the most popular quality of a standard. The preferred 
quantity of figures will be explored first. 

A preliminary step was the formulation of a measurement of 
preference for the pamphlets. The experimental group read twelve 
pamphlets and indicated their interest in three different ways, as men- 

“ See Appendix 11 D. 

“"See Chap. IV, pp. 95-10*. 

“See Chap. V, pp. 137-4*. 
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tioned above. They rated each section on a five point scale from “very 
interesting” to “dull”} they gave a general estimate upon whether 
the pamphlet was “very interesting,” “interesting,” or “dull”} and 
they compared three pamphlets on the same subject with one another, 
ranking them in order of interest. The average of readers’ general 
ratings of every pamphlet was combined in one figure with their 
ratings of all the sections in the pamphlet. These average figures 
for every pamphlet, which were obtained from the group’s general 
and section ratings, were ranked with i signifying the best liked 
pamphlet and 12 the least popular. When two pamphlet averages 
fell at the same place on the scale, they were rated at a half-way 
point} for instance, and G in Table II both ranked at 1.5. The 
resulting scale is called the Pamphlet Preference Index.®® 

The quantity of figures in each pamphlet, expressed in percentage, 
was found by dividing the number of pages containing figures by the 
number of pages in the pamphlet. The percentages in the pamphlets 
were ranked from i to 12, with i representing the pamphlet with the 
highest proportion of figures.®* Table III shows how the rank order 
for proportion of figures was derived and gives the rank order of 
preference for the pamphlets for comparison. 

The interpretation of this table is best made through an average. 
It would be stretching the law of cause and effect to say that the quan- 
tity of figures in the two most popular pamphlets, D and G, was 
therefore the proper quantity to provide in all writing. There are 
more causes for the popularity of those two pamphlets than the 
figures in them. But a careful examination of the last two columns 
of the table will show that the better liked pamphlets, ranking from 
I to 6 in the Preference Index, almost always have the least figures, 
ranking from 7 to 1 1.5 on the scale of quantity of figures. In addi- 
tion, there is only one pamphlet among the half that are less liked 
which has few figures. 

Column I shows that the pamphlets ranking from i to 6 in quan- 
tity of figures have figures on from 95% to 50% of their pages, 
whereas the pamphlets ranking from 7 to 11.5 have from 44% to 
of figures. Therefore the comparison between preference and 
quantity of figures indicates that by and large the pamphlets which 
were more popular with this group had figures on 44 or less of 
their pages. This generalixation is not invariably correct} a most 

^ Each pamphlet is represented by a letter. The key is given in Chap. IV, p. 75. 

“See Chap. IV, p. 88 and Appendix III A. 

"See Chap. V, Table XIV, p, 122. 
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Table III 


Comparison of the Quantity of Figures in Pamphlets with Readers’ 
Preferences for the Pamphlets'*' 


Pamphlet 

Per Cent Pages 
with Figures 
(I) 

Per Cent Figures 
Rank 

(2) 

Pamphlet Preference 
Indexf 
(3) 

Unemployment 

n 

86 

3 

9 

c 

40. 

8 

3 5 

D 

60 

5 

1-5 

War and Peace 

E 

84 

4 

II 

F 

00 

II. 5 

5 

G 

30 

9 

1-5 

Government 

H 

00 

II. 5 

8 

I 

50 

6 

6 

K 

44 

7 

7 

Unstable Money 

0 

95 

I 

12 

P 

21 

10 

3-5 

Q 

89 

a 

10 


* Correlation between figures and preference is —.59. t Tiis Index is based on the 
preference average of 45 readers. J Each letter represents a pamphlet, 

(i) Determined by dividing number of pages with figures by total number of pages in 
pamphlet. 

(a) Pamphlet with most figures ranks i. 

(3) Most popular pamphlet ranks i. For derivation of Index, see Chap, IV, Table VI 
and Appendix III A. 

popular pamphlet, D, had figures on 60% of its pages, and one of 
the less popular, H, had no figures. But it may be safely asserted 
that on the average the proper quantity of figures in reading material 
should be less than 50% in order to satisfy the reader preference of 
the group in this study. 

The preceding discussion illustrated a method of finding the group’s 
standard of judgment by comparing its preference for pamphlets with 
knowledge about the quantity of a topic present in those pamphlets. 
Although the same general method Is used, the details are different 
when the quality of topics is compared with preferences. Readers’ 
attitudes toward the kind of bias expressed by authors in the pamphlets 
will demonstrate the qualitative method. 
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Table IV 

Comparison of Methods of Presenting Issues with Group Preference 
for Pamphlets 



Pamphlet 

Preference 

Index* 


Type of Biasf 


Pamphlet 

Impartial 

a 3 

n Salesman 

Prejudiced 

Partisan 

Unemployment 

B . 

9 

X 



C 

3-5 

X 



D 

1-5 


X ... 


Peace 

E 

II 


... X 


F 

5 



X 

G 

1-5 



X 

Government 

H 

8 


X ... 


I 

6 

X 



K 

7 



X 

Money 

0 

la 

X 



P 

3-5 


X ... 


Q 

10 


... X 



X Shows the consensus of 4 judges’ opinion on the proper position of the pamphlet. 

* The ranks vary flora i to la on a scale of popularity. l is most popular, 12 is least 
popular. The Pamphlet Preference Index is based on the opinions of 45 readers. 

t The types of bias are stated in text above. 

Four ways in which an author may express or may attempt to con- 
ceal his viewpoint toward his subject were distinguished by the writer 
and two assistants. These four types of bias were: 

Position I. Author presents equal facts and arguments on all sides of the issue. 
(Impartial) 

Position 2. Author gives the facts on all sides, but shows why he favors one 
side. (Fair partisan) 

Position 3. Author presents own views in best light, without indicating alterna- 
tives. (Salesman) 

Position 4. Author presents other sides only to show their defects, and good 
points of own views. (Prejudiced partisan) 

Three assistants read each pamphlet carefully and assigned it to 
the one of the four positions that best described the author’s method 
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of presenting a controversial subject. In case the assistants did not 
agree, the writer was the final judge of the position of the pamphlet. 
The position to which each was assigned is shown in Table IV. 
More pamphlets were assigned to the “impartial” position than to 
any other, while only two were considered to be “salesmen” of their 
particular viewpoints. 

It is necessary in studying pamphlets’ bias, as it was in studying 
their quantity of figures, to compare the preferences among a block 
of pamphlets rather than for isolated ones. In this instance the 
average preference for each type of bias is most instructive. The 
greatest contrast is between preferences for pamphlets in types 3 and 
4. The two pamphlets in type 3 were not popular, ranking 9 and 10 
on the Pamphlet Preference Index. Each of the three in type 4 
ranked in the upper half in popularity. The readers liked the pam- 
phlets classified as “prejudiced partisan” very much more than those 
called “salesman.” Furthermore, only one of four pamphlets in type 
I was ranked among the first six in preference, whereas two of the 
three assigned to type 2 were ranked in the upper half. 

There are so few pamphlets spread over so many types of bias 
that it is impossible to consider the results of the examination of bias 
as reliable. It appeared that the readers in the Civil Works Ad- 
ministration group liked most of the pamphlets which employed 
stirring arguments, either fair or unfair, m showing their partisan- 
ship better than they did the pamphlets which were calmly impartial 
or which showed one point of view without admitting opposition. 

Before proceeding to the other considerations of Chapter III, the 
steps in evaluating books which have been illustrated by this study 
should be summarized. The steps to be followed are stated in the 
first sentence of dach of the following three paragraphs. The meth- 
ods which were found by this study to be most practicable in fulfilling 
the steps are given in the remainder of each paragraph. 

It is first necessary to secure a guide to criticism which will include 
all the standards ordinarily erected by readers of literature. Such 
a list was prepared by examining experimental and general studies 
and by asking a group of readers to evaluate material and write their 
criticisms. Several thousand reasons thus obtained were assembled 
in a list of thirty-eight criteria. The list was later condensed into 
fifteen criteria, which may be sufficient for the use of others who wish 
to provide guides to the criticism of controversial literature. 

Next, the reading materials must be studied from several sig- 
nificant angles, in order to discover qualities which cause readers to 
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like or dislike them. The characteristics of the experimental pam- 
phlets which were analyzed in order to illustrate these methods were: 
difficulty of words and sentences, quantity and form of figures, quan- 
tity of history, literary merit, techniques for reaching reader interest, 
format, and method of presenting controversial issues. The quantity, 
the degree, or the type of each quality present in each pamphlet was 
ascertained. Wherever possible, the pamphlets were ranked in 
order upon the degree of the quality that they possessed. 

Then the tastes of the study group are subjected to careful anal- 
ysis. A general picture of the experimental group was obtained from 
a questionnaire. The members ranked pamphlets for preference 
which revealed their average tastes. They rated the material for 
quality of explanations, difficulty, appearance, and several other 
factors. When the preferences of the group as a whole had been 
studied, the tastes of members of the group with different back- 
grounds were examined. 

If an adult leader uses this technique in selecting reading ma- 
terial, he will have an indication of the best book or pamphlet to be 
used during a study course. On the basis of the present analysis, one 
pamphlet on each of the four topics could be recommended for the 
Civil Works Administration group, or a similar one. The selected 
pamphlet in each series might be the one that ranked most popular. 
Or it might be the one that excited the least clash of opinions, fre- 
quently ranking second in its series. The rankings indicated by the 
group would solve the immediate problem of selecting a book for its 
purposes. 

Of more lasting importance than the selection of one book for a 
course is the information acquired about the kinds of books which 
are accepted and rejected by a group and reasons therefor. These 
may be phrased in terms of general standards used by the group in 
judging books. Some of the generalizations drawn from an analysis 
of the experimental group will be stated in the following section. 

RULES FOR SELECTING MATERIAL DERIVED FROM THE STUDY 
OF ONE GROUP 

We have maintained that each careful analysis of readers’ atti- 
tudes toward materials makes the next selection and evaluation of 
materials easier. While rules for selecting materials which are estab- 
lished in one instance may not prove indicative in all cases, the fact 
that they were applied on one occasion makes it more probable that 
they will be applied at other times. Such rules provide a point of 
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departure for the formulation of other rules; every new formulation 
moves the point of departure closer along the road to the attainment 
of adequate standards for selecting curriculum materials for adult 
groups. In this spirit, the generalizations which may be advanced 
from the study of a Civil Works Administration group that read 
twelve pamphlets on controversial problems are stated. 

Readers’ opinions of books provide the standards for selecting them. 
No abstractly determined standards are valid unless they coincide 
with the thinking of readers. Therefore, the opinions which were 
most frequently mentioned in evaluating these pamphlets are of more 
than ordinary importance in selecting books. The readers in the 
experiment stated the following factors to be essential qualities of 
reading materials;^® 

Material should be simple, 

But should not imply that readers are less intelligent than the author. 

An author should seek ways of relating his subject to the daily life and the 

vital problems of the reader. 

Readers are anxious to possess facts, 

Facts which are explained simply. 

Featuring statistics as clearly presented and as easily interpreted as possible. 

Readers are Impatient with wordiness; 

Yet they demand that difficult ideas be expressed fully. 

They are adversely critical of authois whose pleas are based on emotion. 

But they like to have emotional and dramatic touches added to expository 

statements. 

The study discovered that widely divergent sub-groups frequently 
attached the same importance to standards in judging the same ma- 
terials. Difficulty of subject matter, methods of explaining data, 
and methods of presenting facts were important considerations for 
Ph.D. candidates and for non-graduates of college, for men and for 
women, for students trained in the subject matter of the books and for 
students not so trained.®* The differentiation lay in the widely di- 
vergent degrees of each quality demanded by various groups. For 
example, advanced readers were impatient with the illustrations 

“ Derived from analyses in Chap. VI, and from importance ranking in Chap. IV, 
Table V. 

Comparison of the importance of standards according to the judgment of six 
sub-groups, see Chap IV, pp. 95-102. 



AIDS IN SELECTING CURRICULUM S7 

necessary in explaining a subject to beginners. Advanced students 
labeled material “too elementary” which high school graduates de- 
scribed as “not clear.” Nevertheless, manner of explanation and 
difficulty of subject matter were mentioned as influential factors, 
either positively or negatively, by both groups. 

Several qualities of the pamphlets were subjected to a scrutiny 
that revealed some pertinent generalizations for the standards of the 
experimental group. Since a small number of pamphlets was sub- 
jected to a comparatively large number of tests, it is not possible to 
maintain with certainty whether statistics, history, difficulty, bias, or 
other factors were influential in forming readers’ opinions of interest. 
Alternative interpretations, together with experimentation to dis- 
cover which interpretation is more plausible, are given in other por- 
tions of the book indicated by cross references. The following gen- 
eralizations are an attempt to cut through the qualifications and to 
present the findings in a light that seems justified by the data. 
Historical Allusion^'' 

While every pamphlet contained some history, the proportion was 
not great in any of them. The readers tended to like those pam- 
phlets with less historical information, although one with a large 
amount was popular. It is possible for well presented treatments of 
history in a discussion of current affairs to arouse the interest of the 
experimental readers, although material with less historical data is 
ordinarily more interesting to them. 

Presentation of Figures 

The pamphlets which contained figures ranked on a scale from o to 
95 per cent.^® There was a more intense dislike of figures than of 
historical allusions. This fact was shown not only by a comparison 
of ranks for preference and for figures,*® but by a comparison of all 
the pamphlet sections with and without figures.®® Notwithstanding 
the general trend, one of the most popular pamphlets contained a 
large number of figures and some tables.®® 

Analysis of the most and least popular sections of pamphlets showed 
that figures were more successful if their significance were indicated 
than if the reader had to interpret them unaided.®* 

“See Chap. V, p. 134. 

“See Chap. V, p. izz. 

“See Chap V, p. 133, 

“See Chap. V, p. 135. 

“See Chap. V, p. 135. 

“See Chap. V, pp. 155-58. 



S8 BOOKS FOR ADULT STUDY GROUPS 

Difficulty 

The Winnetka formula for determining difficulty, which is a de- 
vice used by the public schools, placed the easiest pamphlet on the 
fifth grade level and the hardest pamphlet on the tenth grade level.®^ 
Although a correlation between preference for pamphlets and their 
difficulty was negligible,^* closer analysis showed that the pamphlets 
that were average in difficulty were more popular than those that 
were either hard or easy.®® 

Readers seemed to have in mind a definition of difficulty that the 
formula did not include, for they ranked only half of the pam- 
phlets in the same order of difficulty as the formula.®® On the basis 
of the difficulty ranking made by readers, the sub-group that had gon^ 
to college two years or less preferred the four easiest pamphlets, 
while the group with postgraduate degrees liked the most difficult 
pamphlets.®'^ 

Dramatic versus Exfository Techniques 

In spite of the fact that the term “dramatic techniques” had a wide 
definition, including examples and comparisons as well as the usual 
employment of stories, it was of much less importance in the minds 
of the readers than were good expository techniques. One of the 
experimental readers’ primary requirements was that information be 
clearly and simply explained, with reasons given to substantiate argu- 
ments.®® 

Techniques of Controversy 

The method which is generally advocated in dealing with issues, 
a sound and impartial survey of all viewpoints, was not in favor 
with the experimental group. This group preferred those pamphlets 
which were strongly prejudiced in favor of one viewpoint, which 
erected arguments of others merely to refute them. The group op- 
posed those pamphlets which argued subtly for one view without pre- 
senting any alternatives.®® 

'“See Chap. V, Table XII, p. 117. 

“See Chap. V, Table XX, p. 133. 

“See Chap. V, p. 134. 

“See Chap. V, Table XIX, p. 13 1. 

°’See Chap. V, p. 138. 

“See Chap. VI, p. 182. 

“See Chap. V, p. 136. 
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Title of Pamfhlet 

The title was by far the strongest reason for preliminary selection 
of books to read.^ 

These few conclusions may be negated by future studies. They 
are presented with the hope of stimulating critics to follow investi- 
gations of their own. Some methods which are a practical outcome 
of the experiment will be presented next. 

APPLICATION OF METHODS FOR SELECTING READING MATERIAL 

The methods which were followed in this experiment in obtaining 
evidence about reader standards of judging literature may be adapted 
to various requirements. The people who can profitably use these 
procedures are a group leader or program chairman in a community, 
a curriculum director of a national organization, a publisher who is 
interested m providing material suited to particular kinds of groups, 
and a writer of study, books. The process of selecting materials herein 
advocated can best be illustrated by considering the work of a pro- 
gram chairman in a community adult education institute. Modifica- 
tions will be suggested for other uses. 

AffUcation of Method by a Local Program Chairman 

The person m charge of curriculum selection must possess pa- 
tience, ingenuity, and flexibility. He must realize that the selection 
of appropriate materials is a dynamic and progressive process and 
will never be completely reduced to formula or routine. He should 
not expect to solve his pi'oblem of establishing generalizations at the 
first trial but should search systematically for means of improving 
the selective process. 

Specifically, the program chairman should become as well ac- 
quainted as he can with the group for whom material is chosen. He 
should note particularly the prejudices which would make selection 
of materials written in one vein dangerous, in another acceptable. 
Many other cues to his selection would be evident, such as the con- 
cepts that the group does not comprehend, or the background of occu- 
pational and cultural experience that it possesses. Such information 
will help to show the broad subjects that should be studied and the 
outer limits of style and complexity that the group would enjoy. 

The general information concerning the group can be obtained from 
those who are familiar with it. Those who know the members best 


" See Chap. VI, p. 187. 
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will find, however, that even they lack specific knowledge of value in 
choosing between several topics or types of material. At this point 
the cooperation of an adult class should be obtained for the purpose 
of studying materials and providing a basis for establishing criteria. 
This study may be profitably carried on by asking a leader to convert 
his class in an introductory course into an experimental unit. 

The first term in which an introductory course becomes the subject 
of an experiment may be devoted to appraising books or other reading 
materials as a part of the class procedure. Several committees of 
members, picked without respect to their literary judgment, may each 
select one piece of reading from among several. These committees 
should be given guides for criticism and should be asked to deter- 
mine, not whether the selection would suit the class — a hard ques- 
tion to answer — but whether it appeals to them individually. 

The books selected by committees should be given to the whole 
class to study during the course. Any comments upon the presentation 
of material, as well as the type of responses that the books produce — 
whether emotional or reasoned, whether quiescent or active — may be 
recorded by the leader. At the end of the course all readers should 
state or write their opinions of the books. In addition to making 
comments on each book separately, they should compare books with 
one another, ranking them for preference and upon other points of 
judgment. 

The leader and program director will have three tasks, if they 
use the results of their experiment to best advantage. First, they 
should analyze the books studied during the course to discover the 
method of securing reader interest, the amount and treatment of 
statistical data, the difficulty, the type and detail of explanations, etc. 
Second, they should scrutinize the readers’ opinions to observe the 
reasons given for liking or not liking the treatment of statistics, the 
attitudes toward devices for catching interest, the quality of explana- 
tion, and so on. Third, they should record important information 
about the class, so that future classes could be compared with this 
one. If other groups are like the experimental group in essential 
respects, they may be expected to like the same type of material} if 
they have essential differences, they would probably prefer other 
material. 

When information from the preceding three analyses is assembled, 
standards may be derived. All of the information about the class’s 
opinion of the experimental material should reveal the qualities that 
they expect in any material of the same nature. These expected quali- 
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ties should be formulated in as definite rules as the data warrant, 
indicating those rules that are reasonably reliable and those that are 
purely tentative. 

An experiment of the type just suggested would have several re- 
sults, One contribution would be the formulation of methods for a 
routine criticism of all materials as classes used them. The immediate 
value of the experiment should be the provision of valuable stand- 
ards for selecting other material for the same introductory course. 
In addition, the standards which were erected by the first class as it 
studied an introductory course may become the starting point for 
analysis of criteria in other courses, either advanced ones in the same 
field or courses in other subjects. If the original experimental class 
is taking an advanced course, it will judge its books by many of the 
standards used before. Some of its standards will have changed: there 
will have been some growth in attitudes and concepts, and it will 
demand different topics. The leader of this group should analyze 
the standards utilized by the advanced class, indicating which have 
remained constant and which have changed. In the same way, the 
program director may request all leaders to compare the standards 
used by their classes with those of the experimental group. 

As each new field is entered by the educational organization, or as 
courses on a new level of difficulty are approached, a detailed pro- 
cedure to establish proper material for study groups is needed. The 
list of important criteria will be modified as books in other areas 
than the experimental one are analyzed. An intensive educational 
campaign may change some of the requirements placed by readers 
upon materials. The shifting political scene may bring new problems 
and stir up a new type of interest in their solution. In spite of 
changing demands by the readers, the task should become increasingly 
easier if general rules of selection are frequently amended to fit 
changing groups, books, and interests. 

When a program chairman has conscientiously analyzed standards 
for several years, he should have a file showing the qualities which 
groups with different backgrounds, at different levels of study, de- 
mand in the various most frequently studied courses. 

Methods of a National Curriculum Director 

The position of a curriculum director at the national headquarters 
of an organization is in some ways less fortunate and in other ways 
more fortunate than that of a local director. It is more difficult for 
him to guess the character of his public, for he does not have the 
personal acquaintance with it that a local leader does. He has a much 
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wider audience, however, and a more varied setting for experimenta- 
tion. Since he may submit the same material to a large number of 
groups, he should be able to establish rules of selection more rapidly 
than a person whose material is used by relatively few students. 

The national curriculum director should enlist the active coopera- 
tion of a few leaders who arc willing to interest their classes in a 
careful evaluation of material that has been prepared or recommended. 
In addition, guide sheets with a list of points on which material is to 
be criticized should accompany every study manual that is sent to 
groups throughout the country. Leaders should be asked to obtain 
an evaluation of the material from the members and to return it to 
headquai'ters. Every sheet of criticism should have attached to it a 
brief questionnaire which is designed to obtain relevant facts about 
the person making the criticism. This is essential if the director is to 
become acquainted with the members. 

When the brief questionnaires from all readers and the detailed 
criticisms from selected groups are returned, the curriculum director 
should perform the same kind of analysis that was recommended for 
local leaders. He should secure all of the information about the 
curriculum material that was obtained in the present study and 
other information besides. He should analyze opinions thoroughly, 
classifying each analysis according to various facts known about the 
person that made the criticism. Then the classifications should be 
tabulated according to ages, occupations, education, or other divisions 
which the investigator found to be influential among his clientele. 

_ This analysis will indicate needs for differentiating material for 
different types of people and will show, more adequately than hearsay 
or Intuition, the standards of judgment used by the members. This 
process should lead to preparation or selection of better material in 
succeeding years. 

Afflkation of Method, by a Publisher 

The business of publishers is to supply readers with books that 
they want to read. They spend money and energy in attempting to 
gauge public tastes. Possibly a file of the kind of information sug- 
gested here might provide a more reliable index to tastes than any 
less thorough methods that they have worked out, 

A recent advertisement shows how one publishing house is seeking 
to analyze reader tastes [i, p. i8]. This illustrates a method that 
might be used in securing reader opinions. The publishers enclosed 
a post card in every copy of Walter Duranty’s I Write as I Please, 
a staff correspondent’s book about Russia. This post card asked for 
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a written answer to the question: ‘‘We would like to know whether 
this book lived up to your expectations. I£ so, why? If not, why 
not?” The only information that was requested about the indi- 
vidual who answered the card was his name and address. They 
claimed that hundreds of cards were returned. Four responses were 
printed in the advertisement. 

The use which this particular publishing house made of the cards 
is not known. If they are to be used as guides to giving the public 
more books that they like, then the responses which were received 
should be classified to show the various standards employed by the 
readers in judging the book. Specific criticisms should be especially 
welcomed, and should be checked against the qualities of the book 
that aroused them. When the publishing house considers other books 
which might be presumed to interest the same audience, it would 
appraise the books in the light of the standards which the readers 
expressed earlier. 

The method of obtaining criticisms which was described in the 
advertisement might be improved in two respects. Two or three 
challenging questions which would insure more specific criticisms 
might have supplanted the single question. And provision for ob- 
taining a few significant facts about the readers would have added 
greatly to the value of the returns. 

If publishers could frequently ask readers why they did or did 
not like a book, assuring the correspondent that the request was made 
in order to give him many better books rather than to advertise a 
single book, they would undoubtedly obtain enough intelligent re- 
sponses to establish more adequate standards for selecting books than 
they have at present. 

Afflication of Method by a Writer 

A writer may utilize some of the methods for becoming acquainted 
with readers suggested in this study as profitably as any other pro- 
fessional worker. If he knows the audience for whom he writes and 
applies his knowledge effectively to subject matter with which he is 
thoroughly familiar, the curriculum director and publisher will find 
their task of selection easy when they examine his manuscript. 

Most of the suggestions that can be made to writers of expository 
material are incorporated in the discussion of the thirty-eight criteria 
around which readers grouped their criticisms of material.^^ ‘Writers 
should become familiar with the comments upon each standard, with 
all that has been found out about them experimentally, and with the 
“See Chap VI. 
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relative importance that they have had for readers. The list of rules 
which are derived from opinions found under the most important 
standards, given on page 56 of this chapter, offers especial help. 

A wide knowledge of the standards that might be derived from 
studies like the present one would provide a good basis for suiting 
material to readers. But that knowledge is not enough. While 
it should be sufficiently mastered by the writer to allow him to comply 
with the general requirements of potential readers, it should not be 
permitted to hamper his expression. Several writers who attempted 
to work with a sheet set before them which coirtained rules for secur- 
ing “readability” requested after a few days that they be permitted 
freedom during writing and that their product be adapted later to 
comply with the rules. Standards are valuable, but they are only a 
partial guide to writing. 

There is no better way for authors who are still learning the tastes 
of their clientele to observe the materiaPs shortcomings and values 
than to present manuscripts to typical people and groups. They 
should record criticisms and derive generalizations from the readers’ 
evaluations. The procedures of some authors who sought to know 
their readers are described in the following illustration. 

Several economists were asked to prepare pamphlets for the boys in 
the Civilian Conservation Corps camps. Before writing, they spent 
some time in the camps, talking to and playing with the boys and 
asking leaders and educational directors about them. If they had 
followed the techniques suggested in this study, they might have 
given the boys printed pamphlets to read and discuss, obtaining their 
specific responses to printed material that was similar to the type which 
the writers were planning to prepare. If, after knowing the boys so 
well, they could have formed a number of groups that would have 
been willing to read and criticize the manuscripts as they were written, 
the resultant pamphlets should have been suited to the needs and 
interests of Civilian Conservation Corps boys. Unfortunately, the 
project of writing the pamphlets for those particular patrons was 
halted before completion, 

SUGGESTIONS FOR FURTHER RESEARCH 

There are many opportunities for investigating reading materials 
for adults. Only a beginning in gathering knowledge about a few 
standards has been made. Other problems will be suggested. First, 
however, a caution about research procedure will be extended. 

Some of the problems presented here can best be solved by labora- 
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tory workers, others by group leaders. Many of them can be in- 
vestigated by either type: the best plan is frequently that of col- 
laboration of research expert and field worker. A warning should 
be given to the research technician working alone. He must not ex- 
periment so carefully with one phase of a problem that important 
conditioning factors are eliminated.^® One reason for the infertility 
of some laboratory products is that the problems studied have little 
relation to actual reading situations. The adult educator is prob- 
ably in no danger of failing to note the whole situation, but he needs 
a warning from the opposite angle. Unnecessary and irrelevant com- 
plications should be excluded from test situations, even at the expense 
of losing a modicum of their naturalness. The mastery of some of 
the basic principles of investigation will save much difficulty when 
the time comes to assimilate and to interpret results. 

Some problems for further research in the field of selecting reading 
matter for study groups are: 

1. The discovery of criteria for fields of reading other than the 
current problems area. What is the relationship between lists of 
criteria obtained from analyzing the comments of a Civil Works Ad- 
ministration group about current problem literature and lists obtained 
from other groups who read other non-fiction materials? 

2. The analysis of causes for interest in different types of popular 
literature: tabloid newspapers, movie magazines, and best sellers 
among books. Coffid any of the sources of interest utilized by popu- 
lar literature be adapted to other purposes? What principles used 
in advertising might be adopted in exposition? 

3. Collection of and experimentation with techniques for increas- 
ing and improving the reading habits of members of groups. 

4. Further testing of the finding here that the group’s judgment 
of a book before reading it generally agreed with its preference after 
reading the book. Are prejudgments about books usually reliable 
for groups? For individuals? Could the selection of study material 
by a group after brief examination of a number of alternatives be 
substituted for the slower method of analysis suggested herein? 

5. Erection of more adequate standards for various criteria in the 
contemporary problems area: 

a. Historical allusions. The conclusion of this study, that his- 
torical allusions have some influence upon reading interests, needs 
further validation and amplification. Do most readers have a preju- 
dice against inclusion of historical data in the discussion of current 

"An illustration of such a study is given in Chap. VI, p. 168. 
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questions? With what presentation is history interesting to people: 
in narrative form, as recital of incidents, as mere allusion to episodes, 
or as illustrating or otherwise relating to present-day problems? 

b. Presentation of figures. The presence of statistics frequently 
prejudices readers against factual articles. How can such articles be 
presented more pleasingly and meaningfully? Which types of graphs, 
charts, and tables are best understood by lay readers? How may the 
average person be taught more effectively to interpret printed figures? 

c. Selection of information. The exclusion or inclusion of infor- 
mation was found to be an important determinant of readers’ prefer- 
ence. Do readers prefer material that is largely familiar to them? 

i How much new information do the following classes of readers want: 
those who possess no background in the field, those who are fairly 
familiar with the field, or those who are experts? If a specific group 
is studied, what are the relations arnong its ability, its previous infor- 
mation upon the topic, and the amount of new information that it 
can master in one reading? 

d. Difficulty. Although William S. Gray is conducting extended 
researches in the field of difliculty, he is preoccupied with the me- 
chanics of construction, and some of the subtler causes for difficulty 
remain to be discovered. With a given background and ability, how 
few definitions and verbal illustrations does a person need in order to 
learn a new idea? How many ideas can be crowded into a paragraph 
without confusing the readers? What are the best forms for defini- 
tions of words and statements of problems? 

e. Colloquial style. The advantages and disadvantages of writing 
In the language of the reader for whom material is intended need 
more impartial consideration than they have yet had. Humorists 
have gained wide fame through talking in colloquial language j what 
would be the effect of wider use of their style in popular discussions 
of serious questions? 

f. Dramatic versus exfository techniques. The evidence of this 
study suggests that dramatic techniques have been overrated as meth- 
ods of interpreting facts. How do the dramatic and expository styles 
compare (i) when the reader has no previous interest in a subject, 
(2) when the interest is mild, or (3) when the reader is anxious to 
acquire information? How does the nature of the reader and how 
does the content of the material influence reader attitudes toward 
dramatic or expository methods? 

g. Illustrations. Much more attention should be paid to the in- 
structional value of photographs and cartoons in non-fiction material. 
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What is the difference between the quality of learning through un- 
illustrated context and through context which employs pictures?^** 

h. Introduction md conclusion. An introduction has the power 
of orienting readers favorably or unfavorably toward a book. A 
conclusion determines to some extent the impression left by a book. 
What constitutes successful introductions and conclusions? Do people 
want their attention arrested by stories, or do they want concise facts 
at beginning and end? What influence does familiarity with a topic 
have on the required nature of an introduction? What is the influ- 
ence of the length of the article or book on the type of conclusion 
necessary to satisfy the reader? 

i. Techniques for catching the reader's attention. Although al- 
most any study of readability involves some analysis of techniques 
for interesting readers, there is need of direct attention to the prob- 
lem. What methods of relating problems to readers’ interests are 
used in successful and in unsuccessful writing? Are there variations 
in the methods preferred by different readers, or by the same person 
when reading about different subjects? 

j. Prejudice. Sociological methods should be employed to dis- 
cover the wide influence of the reader’s or the author’s prejudices 
upon readability. Under what conditions do readers like material 
which is opposed to their beliefs? Can a calm author persuade 
readers to consider ideas that are distasteful to them? Does an emo- 
tional author induce emotion in readers? What is most successful: a 
strictly impartial presentation, one in which the author is fair but 
partisan, or one in which the author is so partisan that he admits no 
advantages in other views? \ 

SUMMARY 

This study formulated three steps in procuring adequate bases for 
standards to be used in selecting an adult education curriculum. First, 
the qualities that readers criticize in books were discovered by classify- 
ing all of the experimental readers’ opmions of pamphlets under the 
heads of thirty-eight qualities. Then a few of these qualities were 
selected to ascertain their form in the experimental pamphlets. 
Third, the scales of qualities resulting from the preceding analysis 
were compared with a readers’ scale of preference for the pamphlets. 
This comparison showed the qualities that readers liked best in the 
pamphlets. 

“The magazine, Building America, is using illustrations as a basic method of 
teaching. [2] 
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The information provided by these three kinds of investigation 
was organized to show the standards that the experimental readers 
used when they evaluated the pamphlets, and therefore the standards 
that they could be presumed to use in selecting any material of the 
same general nature. The standards were stated in terms of the 
amount or the characteristics of each quality in the pamphlets. 

The purpose of this study was twofold. In addition to accumulat- 
ing information about standards to use in selecting books, methods 
that could be utilized in selecting or evaluating books were evolved. 
A local program chairman was advised to discover the standards ap- 
plied to their reading by groups under his control through a three- 
step analysis of the sort used in this study. A national program 
director was expected to apply a similar method of accumulating 
criticisms from readers throughout the country to the material that 
was prepared under his direction. Publishers, too, could profit 
through the criticisms of their books by readers if they could find a 
method of obtaining a representative cross-sampling of opinion. 
Finally, authors should submit their manuscripts to groups represen- 
tative of potential readers in order to shape the writing to the re- 
quirements of those readers. 

Enough information about the various standards for selecting books 
was acquired here to make the possibility of fruitful exploration and 
the need of far more research in the field self-evident. More than 
a dozen research problems in the same area as this study were 
mentioned. 

Chapter III has touched upon most of the content of Part II. 
Part II contains a more elaborate report of the investigation, stating 
in detail the methods used, the difficulties encountered, and the com- 
position of data from which the few findings were drawn. In Chap- 
ter VI the threads of the discussion are brought together to show all 
that was discovered about each of the standards used by readers in 
judging pamphlets upon current controversial issues. 
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CHAPTER IV 


TECHNIQUES OF THE INVESTIGATION 

The detailed considerations of performing an experiment form the 
main part of this chapter. The understanding of such procedures as 
classification of standards and methods of studying responses within 
the sub-groupsj described in this chapter, is essential to an intelligent 
assimilation of the discussions of Chapters V and VI. 

COLLECTION OF A LIST OF STANDARDS 

The first procedure was assembling a working list of standards of 
judging literature. There were four sources for these standards: 
first, the opinions of experts, and second, third, and fourth, criticisms 
by readers of pamphlets In the form of general criticisms, section 
reactions, and pre-judgments of books. A Combined List of Criteria 
was derived from the four sources. 

The Ofimons of Exferts 

In an effort to collect available opinions upon standards for select- 
ing reading material for adults, the books published before the date 
of the investigation^ which deal with adult curriculum, with the gen- 
eral literature of adult education, and with the critical writings of 
publishers were consulted. In addition, some books in the field of 
formal education were included in the survey. Seventeen people 
were interviewed, all of whom were experienced in the selection or 
preparation of reading materials for adults. They included pub- 
lishers, librarians, and specialists in a variety of adult programs. The 
constant query which guided both reading and interviewing was : What 
are the standards for selecting or preparing reading materials for 
adult study groups? 

Of all the sources consulted, only fifteen experts and ten books 
mentioned standards for selecting reading materials. There were 
four instances of an exhaustive formulation of standards by leaders 
who dealt with the selection of adult curriculum materials. In the 
other instances the references to standards were incidental. 

^The examination of literature from which expert opimons were derived was 
completed by the writei in June, 193+. 
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In all, seventy different expressions of opinion regarding the rules 
for selecting material were obtained from interviews and literature. 
They may b^e classified under the following heads. According to the 
experts, non-fiction reading material for adults should. 

A. Suit different needs of readers 

B. Aim to catch and to hold the interest of readers 

C. Possess literary merit 

D. Possess soundness and educative merit 

E. Appear in attractive format 

F. Be adjusted in difficulty to the comprehension of readers 

These headings are expanded with sub-heads and quotations in Ap- 
pendix I A. The frequency with which each was mentioned by 
twenty-five sources is stated, and the classification to which each is 
assigned in the Combined List of Criteria is indicated there. 

The criteria gained from reading and interviews depended largely 
upon unvalidated opinions. Program builders had not made an ex- 
perimental approach to the problem of setting up complete standards 
for selecting reading material j the criteria represented rather the 
impressions and comments of people who considered the selection of 
curriculum content to be a minor aspect of their adult education ad- 
ministration. 

Nevertheless, the criteria obtained from the opinions of experts 
have a value. They have been developed from experience, and such 
opinions are sometimes wiser than generalization obtained through 
formally established techniques. An editor told the writer that it was 
a better test to submit any manuscript to a certain executive who had 
traveled up and down the country supervising the organization to 
which its potential readers belonged than to give it to several trial 
groups to read and criticize. The rich experience of a man who has 
lived with manuscript, as had a newspaper and magazine editor like 
Walter Hines Page, offers valuable data. 

Formulation of an Exferiment 

Since the examination of expert opinion gave no ready answer to 
the question of how to select study material for adults, the informa- 
tion had to be secured elsewhere. It was thought that the best way 
to learn how to select materials was to observe the standards that 
adul ts used in selecting their own material. Accordingly, an experi- 
ment in which ordinary readers responded to real literature was 
planned and executed. 
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Selection of Experimental Reading Material 

The experiment was conceived in terms of requesting adults to 
read and to criticize material of the type that might be used in a course 
of study. The first problem to be faced was the selection of experi- 
mental material. 

It was obvious that samples of the curriculum content from all 
fields of adult education could not be included in the experiment. 
Conversely, it appeared wise to use fairly uniform content, so that 
the readers would react to the way in which material was presented 
rather than to its subject matter. For example, an ambitious sales- 
man with two books before him might eagerly read the one telling 
him how to increase his sales, no matter how it was written, whereas 
a most alluring treatise on the growing of better corn might leave 
him untouched. But if he were given several books on salesmanship 
or several on corn growing, he could judge which he liked best upon 
grounds that would reflect more truly the standards he uses than if 
the single powerful criterion of interest in the subject were allowed 
undue weight. In view of these considerations, the investigation 
was limited to reading material in a single field. 

Which field should become the center of exploration? Much of 
the present impetus is given to adult education by those who believe 
that common citizens need to become intelligent about civic problems. 
They feel that one way of developing social intelligence is to en- 
courage wider reading upon social issues. However, the findings of 
the Waples-Tyler study show the difliculties in the way of persuading 
people to read about social problems. [3, p. 289*] In their study 
“common citizens” rated ten subjects in the current problems area. 
They considered nine of these to be of less than average interest. 

An educational campaign is beginning to meet this lack of interest 
toward acquiring a social education in several ways; by holding forums, 
by broadcasting, and through movies. Nevertheless, these attempts 
are not sufficient. Information about ways to stimulate reading and 
about the selection of material for an educative social purpose grows 
in importance. Therefore, the area selected for this study was that 
of current social, economic, and political problems. 

What should be the form of the experimental material? We 
could have selected books, magazine articles, or pamphlets. Books 
are too long to be used successflally in an experiment involving com- 
parison of various methods of presentation. Magazine and news- 

* Throughout this chapter numbers in brackets refer to bibliographical references 
to Chapter IV, as given on p. 114. 
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paper articles have developed a style of their own not typical of 
that used in more extended curriculum materials. Consequently, 
pamphlets seemed to be the best experimental material. They are 
long enough to have many of the desirable features of a book: an 
introduction and summary, full development of several points, and 
detailed information. At the same time they are short enough to 
leave a unitary impression and to permit one reader to peruse and 
to comment on several in a working day. 

In addition, pamphlets acquire an importance on their own merit. 
They can be produced inexpensively j they arc necessarily simple in 
their statement of issues 5 and they are adapted to brief group dis- 
cussions. If cleansed of the taint that they have earned through long 
association with propaganda, they might well bear the supplementary 
relation to an adult curriculum that work-books do to the text in 
elementary and secondary schools. These considerations justified the 
selection of pamphlets for the experiment. 

Several factors guided the selection of specific pamphlets. None 
was included which was either so technical or so poorly prepared 
that it would never be selected by a group leader in his search for 
good material. Each pamphlet was introductory rather than ad- 
vanced. Further, it was typical of material written for the general 
public, prepared to reach audiences who had not acquired much 
background for study. It is for just such readers that new, more 
interesting books must be written. Only those pamphlets were se- 
lected for the experiment, therefore, that purported to be popular 
introductory treatments. 

Following these preliminary considerations, the chief concern was 
to obtain pamphlets on each topic sufficiently similar in content to 
counterbalance preferences due solely to greater interest in topics. 
When similar content was obtained, wide variations in style and 
manner of presentation of the subject were desired. But the different 
series on each of the topics contained pamphlets with similar style or 
other quality, in order to make the series comparable. Each series 
had one simple and one difficult pamphlet, one that made a straight 
factual presentation and another that enlivened facts with dramatic 
techniques. 

Ten topics were selected, each to be represented by three pamphlets. 
Pamphlets which fulfilled all requirements were found on only four 
of the topics. The selected material was drawn from pamphlets in a 
collection of Paul Hanna, at the Lincoln School Laboratory, pam- 
phlets gathered by John Herring for the Committee on Economic 



TECHNIQUES OF THE INVESTIGATION 7 S 
Policy, and those in the Rand SchooPs bookstore o£ socialist 
literature. 

The titles of the chosen pamphlets follow. The letter preceding 
each title is the key which will be used throughout the book to desig- 
nate that pamphlet. 

Key Topic— UNEMPLOYMENT 

B “Unemployment Relief and Public Works,” Leon C. Marshall, Bureau of 
Publications, Teachers College, Columbia University, New York, 1933, 
National Crisis Series. 

C “Uncle Sam and Unemployment,” Pauline E. Thompson, Bureau of Pub- 
lications, Teachers College, Columbia University, New York, 1933, Na- 
tional Crisis Series. 

D “Unemployment and Its Remedies,” Harry W. Laidler, Socialist Party of 
America, New York, 1 931. 

Topic— WAR AND PEACE 

E “Who Pays for War?” Maxwell S. Stewart, American Education Press, 
Columbus, O, 1933, Modern Problems Series. 

F “The Fight Against War,” Albert Einstein, John Day Co., New York, I 933 j 
J ohn Day Pamphlets. 

G “War Resistance,” William Floyd, Wai Resisters League, New York, 1933 - 
Topic— CHANGING GOVERNMENTS 
H “Roosevelt and the Constitution,” Selden Smyser, American Education 
Press, Columbus, O., 1933, Modern Problems Series. 

I “Modern Economic Systems,” Gertrude Wolff, American Education Press, 
Columbus, O., 1933, Modern Problems Series. 

K “The Yankee Primer,” Oscar Ameringer, The American Guardian, Okla- 
homa City, Okla., 1933. 

Topic— UNSTABLE MONEY 

O “Inflation" What Is the Gold Standard i*” H. Gordon Hays, American 
Education Press, Columbus, O., 1933, Modern Problems Series. 

P “Unstable Money,” John Strachey, John Day Co., New York, 1933, John 
Day Pamphlets. 

Q “Inflation and Your Money,” Edward H. Collins, Duffield and Green, 
New York, 1933. 

Each pamphlet was subdivided into sections suitable in length and 
unity of thought for analysis by the readers. In some cases the sec- 
tional divisions made by the authors or editors were used, but in 
others the editorial sections were of such uneven lengths that it was 
necessary to divide or to combine them. 
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Set-uf for Obtaining Reader Criticisms 

The necessity of finding adults who had enough time to give a 
report of their reasons for liking or disliking material was a real 
problem. Although several adult groups proffered their help, 
volunteer assistance is precarious if an experiment involves perform- 
ing painstaking analyses without the incentive that may be offered by 
a leader who is known to the group, 

A solution was unexpectedly obtained when the opportunity arose 
to get governmental aid. The time of forty-five people working 
under the Civil Works Administration auspices in New York City 
was granted for the experiment. At first there were ten workers re- 
porting over several weeks, who not only read and criticized the 
pamphlets, but who also served as assistants to analyze the pamphlets 
and the opinions of the other workers. A further group of thirty- 
five read and wrote comments five hours a day for four days. 

Each of the readers was given one series of three pamphlets at 
a time. Accompanying the pamphlets were a general instruction 
sheet (Appendix II A), instructions for rating sections (Appendix 
II B), a form on which to rate the sections (Gi'aph i, page 78), and 
a plain sheet of paper on which the comparison of three pamphlets 
was to be written. 

Verbal instructions emphasized the contribution that the readers 
could make by being honest in their efforts to analyze and to state 
their reasons for interest or lack of interest m the literature. The 
purpose of the study was explained in terms calculated to enlist their 
interest and cooperation. Without attempting to circumscribe the 
readers’ statements too much, the instructions asked them to give 
searching rather than superficial reasons. For example, the comment, 
“This is interesting because it tells about an unemployed family,” 
was denounced as giving no real reason for the in|erest, and the 
criticism, “This is interesting because it dramatizes facts,” was substi- 
tuted because it was more exact. 

Since uniformity of procedure was desired, even though wide 
latitude in the type of possible response was given, the instruction 
sheet was read to the group point by point. Errors or omissions by 
the readers in carrying out directions were noted as soon as the com- 
pleted forms were returned, and individuals were helped privately 
to see their deficiencies. The guidance of the first ten people re- 
porting for work separately was more satisfactory than that of the 
thirty-five reporting en masse. Since their chief error was occasional 
omission of ratings, tabulations did not always total forty-five. 
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When a series of three pamphlets on the same topic was first 
placed in his hands, each member of the reading group wrote on 
the plain paper the title of the pamphlet he would like to read first, 
and told why. He also ranked the titles and covers in the order of 
his preference. These criticisms are called ‘‘snap judgments.” 

Next he read his first choice, with the section rating sheet face 
down beside him. After he had scanned the pamphlet at his ordinary 
reading rate, he wrote a general estimate on the back of the sheet. 
He was given an opportunity to register his judgment of this pam- 
phlet by marking whether it was on the whole “very interesting,” 
“interesting,” or “dull.” This ranking is a major component of the 
Pamphlet Preference Index. 

Then he read the pamphlet through slowly, grading each of its sec- 
tions on the face of the sheet, which was ruled in column scales. If 
the section stood high in his estimation, he checked “very inter- 
esting,” and for lesser degrees checked “rather interesting,” “inter- 
esting,” “slightly interesting,” or “dull.” The check was placed be- 
side the number of the section, which was printed on the sheet. On a 
wide column to the right he wrote the reason for his judgment. A 
sample rating sheet, with the section criticisms, is given in Graph I 
on the following page. 

After each pamphlet was evaluated individually, ail three pam- 
phlets on the same topic were compared by the reader. The reader 
was first asked which pamphlet was the most interesting and why. 
He was then requested to rank the three in the order of his preference. 
The reasons given in answer to the question were haphazard or repe- 
titious and afforded no new leads 5 these reasons were not tabulated. 
The rankings for preference, however, were utilized often when pam- 
phlets were compared. 

Finally, the reader ranked the three pamphlets on these six quali- 
ties: attractiveness of print and make-up, explanation of tables and 
numbers, literary quality, difficulty, explanations, and relation to per- 
sonal interest. The judgments on these points are a useful addition 
to pamphlet analyses. 

Chsslficmon of Reader Comments 

One worker copied on cards all the different criticisms of the sec- 
tions of pamphlets. When a comment appeared two or more times, 
a tally mark indicating its reappearance was made on the first card. 
Each card contained only one aspect of a quality. If a comment 
‘ consisted of several aspects, each was placed on a different card. The 
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comment was marked plus if it represented a favorable opinion, and 
minus if it represented an unfavorable opinion. 

When criticisms of slightly more than half the pamphlets had been 
recorded, the clerk announced that few new reasons were appearing. 
She continued to tabulate for two more days, finding only twelve 
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new reasons. On the last day of the experiment reasons were tabu- 
lated for frequency only. Ninety-seven per cent of the different 
reasons for liking or disliking the literature had been given by the 
time sixty-one per cent of the pamphlets had been read. Or, the 
forty- five readers had practically exhausted their variety of judgment 
after commenting on seven pamphlets each. 
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The stream of reader reactions brought an immediate need for 
classifying them. The cards containing similar reasons were placed 
together. By the end of several days enough different responses had 
been obtained to necessitate serious effort in grouping them under 
categories. Three workers, including the writer, classified all the 
cards, compared classifications, and achieved a common form with 
topical headings and subdivisions. The form contained many over- 
lappings, since different interpretations of the same phrases were 
recorded under several heads. This form was satisfactory in handling 
the data as they appeared. A more orderly classification was needed, 
however, before other investigators could use the list as a guide in 
cataloguing criticisms. 

The writer re-ordered all the original data with changed major 
divisions. The new and final organization contained, in addition 
to the classification of the section reactions, all the responses obtained 
from general criticisms of pamphlets, from snap judgments, and from 
experts’ opinions. It is referred to throughout as the Combined List 
of Criteria. This complete list contains thirty-eight criteria. 

The Combined List was developed from the classification of the 
section reactions. Responses from the other three sources of criteria 
were classified under the Combined List, with occasional adaptations 
to suit new expressions of opinion. In order to assign the general 
criticisms to the Combined List, two workers allocated each opinion 
to appropriate headings. The first worker tabulated frequency of 
opinions under each heading, the second wrote the classification beside 
the criticism on the original general criticism sheet, and re-tabulated. 
Although the proper classification of each separate criticism was not 
checked, agreement upon each classification must have been close, 
since the final rankings for frequency of occurrence were the same. 
The general criticisms were so similar in phrasing to the section reac- 
tions that no separate listing of them has been made in this report. 
Samples of general criticisms of two pamphlets are given in Chapter 
V. The frequency of mention of each standard, in terms of the Com- 
bined List of Criteria, is stated in Appendix I C. 

The snap judgment criticisms for each pamphlet, in which readers 
explained why one pamphlet looked more interesting than the others, 
were written on a large sheet of paper. They were likewise classi- 
fied by two workers under the headings of the Combined List of 
Criteria j they were allotted to only seventeen of the thirty-eight 
heads. The frequency of each criticism and its classification may be 
seen in Appendix I B. 



BOOKS FOR ADULT STUDY GROUPS 


The thirty-eight standards in the Combined List of Criteria are 
given in Table V at the end of the next section. 

Comparative Imfortmce of Different Standards 

Up to this point, the discussion has emphasized the need of obtain- 
ing as complete a list as possible of the readers’ reasons for liking 
and disliking given reading material. When a long list was obtained, 
it was evident that all the criteria were not equally important. The 
next problem, consequently, was to discover which criteria were the 
most important. 

The meaning of importance in respect to criteria is furnished in this 
study by statistical methods. That criterion is most important which 
is most often mentioned by readers. Some psychologists object to 
this definition of importance, on the ground that people do not always 
recognize the fundamental, half-conscious reasons for their prefer- 
ences. In this particular kind of investigation that objection is not 
valid. We want to know what people think or say is wrong with 
books. Their verbalized opinions are the standards which they use 
in judging literature. For instance, a piece of material may draw the 
criticism repeatedly, “This has too many statistics.” It becomes the 
author’s task to find why his presentation of statistics is onerous to 
readers. To relieve the bad impression he may rearrange or reem- 
phasize the figures rather than remove them. No matter what the 
basic trouble with the figures, the most important element for the 
readers is the reiterated one of “too many statistics.” They will not 
like the material until they believe that the objectionable feature is 
eliminated. 

The opposite of this statement, that infrequency of mention indi- 
cates lack of importance, is of course not sound. A reason does not 
lack importance because it has not been named. There are doubtless 
not enumerated in this list some factors which have a part in deter- 
mining judgments. Certainly all the criteria mentioned here affected 
attitudes toward the pamphlets j otherwise they would not have oc- 
curred frequently enough to be placed on the list. None of them can 
be ignored, therefore, in the selection or preparation of books on cur- 
rent problems. Infrequency of mention means only that the impor- 
tance of the standard is less. 

In accordance with the concept that frequency of appearance of spe- 
cific standards is for practical purposes synonymous with importance, 
rankings of i to 38 were established for each of the four lists of 
expert opinions, section reactions, general criticisms, snap judgments. 
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Since each list was compiled from a different viewpoint, the rank- 
ings naturally did not coincide. When readers made general 
criticisms of pamphlets they mentioned difficulty of concepts most 
often. When they made detailed comments upon sections of pam- 
phlets, they mentioned most frequently expository techniques. The 
title was the most important determinant for the selection of a pam- 
phlet under snap judgments. When experts were Setting standards 
they mentioned most frequently the necessity for techniques for re- 
lating subject to reader. Thus the four different methods of eliciting 
standards gave different emphases. 

The advisability of combining the four lists into a single importance 
ranking was considered. Viewed from one angle, an average cloaks 
the divergencies of viewpoints, for it does not do complete justice 
to any one angle of the problem, and it may obscure some vital 
factors. For example, the topic of expository techniques ranks i in 
one list, and 9.5, 11.5, and 19 in the three other lists composing the 
Combined List of Criteria. An average does not reveal as plainly as 
the results of the section reactions how vital expository techniques are. 

On the other hand, anyone confronted with four lists of from 
seventeen to thirty-eight criteria each will appreciate the need of 
simplification. An average smooths discrepancies and throws com- 
mon elements into relief. It shows clearly the points upon which the 
majority agree. For instance, it is not impressive that techniques for 
relating subject to reader are considered important by someone. But 
the statement Is heeded more respectfully when four methods of 
evaluation place this standard in the highest quarter of importance. 
An average tends to balance automatically the varying impressions 
that must be adjusted intellectually if this statistical aid is not 
furnished. 

In view of these arguments on both sides, the frequency of stand- 
ards in the four different lists were averaged j but the averages were 
interpreted liberally. The rank orders of each standard in the four 
lists were added, and these resulting sums were ranked for im- 
portance.^ 

The thirty-eight standards by which readers judged the pamphlets 
are listed In Table V, In the general groupings of content^ form^ 
catching and holding reader interest, and dealing with controversial 
issues, there are fourteen main heads, those denoted by Arabic num- 

The fiequency of each standard in expert opinions is given in Appendix I A-z, 
of snap judgments in I B, of general and section criticisms in I C. The method of 
formulating the combined ranks from the ranks of the four components is shown 
in Appendix I D. 
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Table V 

Combined List of Criteria for Judging Non-Fiction Literature, Ranked 
for Importance* 


Criterion 

Frequency- 

Rank 

Quarter 

Placement 

A. Content 

I. Historical allusions .... 

30 

4 

2 . Presentation of figures 

6.5 

I 

q. Treatment of subject 

a. Timeliness 

15 

2 

b. iSelection of information 

6 5 

I 

c. Quality of information . . 

3 

I 

B. Form 

I. Difficulty 

a. Terms 

ao 

3 

b. Concepts 

2 


c. Abstractness 

26 5 

3 

d. Condensation . . 

8 

1. Literary quality 

a. Organization 

18 5 

2 

b. Form and grammar .... 

25 

3 

c. Vitality 

4 

I 

d. Rhythm and fluency . .... 

28 

3 

e. Concise, simple language 

13 

2 

f. Adult approach .... 

36 

4 

3. Literary techniques 

a. Dramatic techniques 

10 

2 

b. Expository techniques .... 

5 

I 

c. Figures of speech 

38 

4 

4. Format 

a- Type 

22 

3 

b. Size of book . . 

23 

3 

c. Cover and illustrations 

14 

2 

d. Footnotes, index, and table of contents 

35 

4 

e. Sections and captions 

3^ 

4 

f. Title of book . 

21 

3 

g. Make-up of book . . . 

32 

4 

C. Catching and holding reader interest 

I. Introduction 

a. Outline of plan 

33 

4 

b. Awakening interest 

17 

2 


* The frequencies upon which the ranks for importance are based are given in Appendix I. 
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Table V {Continued) 


Criterion 


Frequency Quarter 
Rank Placement 


2. Conclusion 

a. Summary of book 

b. Outcome of reader interest i 

3. Techniques for relating problem to reader 

4. Emotional appeal . . 

5. Use of famous author’s name 

D. Dealing with controversial issues 

1. Author’s display of own viewpoint 

a. Non-partisan 

b. Fair partisan .... 

c. Prejudiced partisan . . 

2. Techniques of controversy 

a. Presentation of argument 

b. Criticism 

c. Influence on reader 


erals._ These fourteen heads are the center of the research reported 
later in this chapter and in Chapter V. 

The frequency rank to the right of each standard is a combination 
of four ranks, as explained above. The four components upon which 
the ranks are based did not agree well enough among themselves 
to permit the designation of one point on a thirty-eight point scale to 
have significance} this frequency rank is too detailed to be valid. 
Therefore, the last column of the table, showing the quarter of the 
rank distribution into which each standard falls, is a more honest 
picture of importance than the frequency rank. 

A thorough knowledge of the nature of the nine most frequently 
mentioned criteria and of their application to material written for dif- 
ferent groups would give a writer most of the help he demands for 
preparing popular material. The subsidiary criteria in the second 
and lower quarters of the importance list contribute to a more thor- 
ough evaluation of the material. Even though the topics under con- 
troversial issues are placed in the second or the fourth quarter, they 
are valuable keys to handling certain types of current literature. 
Most of the standards under jormat are in the third or fourth quar- 
ters} yet the fact that the cover, illustrations, and title played a 
definite role in the formation of snap judgments concerning the 


H 3 

11 2 

I r 

9 I 

26.5 3 

12 2 

34 4 

29 4 

16 2 
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pamphlets should not be overlooked. In the same way, the function 
of each of the other criteria can be indicated, as will be shown in 
Chapter VI. 


STUDY OF EXPERIMENTAL GROUP 

A General Picture of the Growp 

Unless there is at hand considerable information regarding a group 
for which books are to be chosen, an unlimited number of general 
principles of book selection and a thorough knowledge of the make-up 
of books are of little value. To illustrate; All of the canons of literary 
taste state that Hamlet is a masterpiece, yet no teacher of an average 
fourth grade would require her pupils to read it in its original form. 
Of course, the teacher accepts literary principles and believes that 
Hamlet is a work of art, but she knows her group’s limitations in 
education and experience. The more an author or leader knows 
about a group of potential readers, the more elScient becomes the 
writing or selecting of appropriate material for its use. One conces- 
sion to this fact has been made in the acceptance, in theory at least, 
of the principle that audiences have varying abilities which must be 
recognized in the use of words and concepts. But more than mere 
word shifting is necessary if literature catches a group’s imagination 
and answers its needs, as well as provides for its vocabulary limita- 
tions. If a leader’s choice of material is to be wise, he must know 
the group’s prejudices, its interests, its allegiances, and its large and 
small problems. 

The Civil Works Administration group may be considered fairly 
representative of other adult education groups. It contained a mis- 
cellaneous assortment of individuals such as one would find in many 
classes. Although the group did not share that common interest in 
studying which brings most classes together, its members had in com- 
mon both the experience of having been unemployed over a number 
of months, and a college level of education. However, the analysis 
of the group is not intended to present a typical picture, but rather 
to indicate one method of discovering some of the factors back of 
certain preferences for materials. 

In this study the experimental group was analyzed as if it were 
an adult education class for which fiirther material was to be chosen. 
A questionnaire,® which was answered m detail, was submitted to 
the members. Each question about the members that was asked was 
based either upon a general factor that other experimenters had con- 
sidered to be important, or upon specific factors that might determine, 
’ The questionnaire is given in Appendix II D 
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to some extent, their interest in this particular set of pamphlets. The 
general questions dealt with facts in their experience: age, sex, edu- 
cation, positions held, religious and political affiliations, and marital 
status. The specific questions concerned education in the areas cov- 
ered by the pamphlets, regular reading habits, and interest in the 
four topics around which the experimental literature was organized. 

Three of the important facts about people which influence their 
outlook on life are sex, age, and marital status. Most of the members 
of the Civil Works Administration group were young unmarried 
men. In the group of forty-five there were thirty-three men and 
twelve women. Thirty-nine members were unmarried. Thirty were 
in the young adult age range between 21 and 27. Two were between 
18 and 20 3 three were over 45 years of age. 

A key to mind sets and loyalties may be found in religious and 
political affiliations. Twenty out of forty-five members replied to a 
question concerning church membership, “None.” There were seven 
Catholics, seven Protestants, and eleven Jews in the group. Many 
of them evaded political commitment: twenty-four checked “split 
ticket.” There were two Republicans, eleven Democrats, six Socialists, 
one Fascist, and no Communists. The interests of the workers were 
more radical than the returns showed. One member of the group 
regularly read The Daily Worker, a Communist paper, and several 
were active in unemployment demonstrations. There was obvious 
approval of radical pronouncements in the pamphlets. 

Economic status is frequently an index to interests. Ordinarily 
an estimate of economic status may be derived from knowledge about 
occupations. The fact that the Civil Works Administration workers 
were accepting relief made their present economic status obvious. 
Questions concerning their vocational training and the types of jobs 
they had held indicated the economic group to which they had previ- 
ously belonged and helped to identify their interests. Most of the 
workers had been trained for professional or business occupations and 
had held jobs in the fields for which they were trained. They 
noted long lists of other interim jobs, such as cashier, druggist, usher, 
promoter, and salesman. 

Ability in the use of language is ordinarily judged by the amount 
of education. This is a rough substitute for the more exact measure- 
ment of reading and intelligence tests. In the Civil Works Admin- 
istration group, all but one participant had been to college. There 
were twenty-nine college graduates, and eight who had postgraduate 
degrees. Thirteen of the group had taken no courses in the social 
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sciences with the exception of high school economics. Twenty-five 
had courses in sociology, thirty-six in economics, and twenty-two in 
political science. Fifteen members had studied one or more courses 
in all of the above fields. 

Attitudes toward specific reading material may depend in part 
upon the amount and kind of reading that people are accustomed to 
do. Questions concerning reading done by the group elicited the 
following information: Although sixteen people had read no books in 
the previous month, the remaining thirty-one reported a total of 
eighty-five books read dunng that period. All of them read news- 
papers. The returns showed that sixty-six magazines were read by 
thirty-seven persons. Twenty-four different magazines were men- 
tioned. Eighteen of the group had read one or more pamphlets in 
the previous year. 

Finally, it is valuable to know the degree of interest group mem- 
bers have in the material that is to be studied. Each person was 
asked to check the four topics under which the pamphlets were 
grouped to indicate the ones he was already interested in reading 
about, and then to rank all four topics in order of their probable 
interest. Only three in the group said that they were not interested 
in any of the four topics listed. Eleven were interested in all of the 
topics. Before the experiment, the topic of war and peace was con- 
sidered most interesting, and unstable money least interesting. At 
the end, unemployment moved to first place, and unstable money 
remained last. (See Table X, page 103.) 

The analysis of the questionnaires made it possible to interpret 
many responses to the pamphlets. The radical tendency of some 
members made the espousal of the radical pamphlets appear natural. 
The conservative educational and professional background of others 
explained one reason for the violent division of opinion upon a so- 
cialistic and a pacifistic pamphlet. The dynamic title of Fight Against 
War would appeal to young people and was one of the most popular. 
Since everyone had been to college, the frequent comment of “too 
elementary” beside many sections was not surprising; on the other 
hand, dislike of statistics was unexpectedly strong. If adult edu- 
cators have similar information about their groups, it may aid in in- 
dicating some subjects of study and styles of writing which would 
probably be acceptable to the students. 

The experimental group did not respond to the literature as a 
unit. Some'^ of the knowledge about the group was analyzed to find 

‘All of the information obtained in the questionnaire about the group was m- 
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explanations for the similarities and the differences of opinions. F or 
instance, was there any difference in response between the men and 
the women in the group? Did people who had had training in social 
sciences think somewhat alike upon these pamphlets dealing with 
social questions? Did their opinions differ widely from those of 
people without such training? Answers to such questions helped to 
clarify the problems involved in formulating general rules for select- 
ing reading material. 

In a search for general trends the responses of the group as a 
whole will be presented. Then preferences of random halves of the 
group will be inspected, in order to find how reliable the general 
trends might be considered. Finally, the preferences and standards 
of judgment of parts of the entire group will point to more specific 
rules for selecting reading material. 

An Index to Preference for Pamfhlets 

Before the responses of the group or any part of it could be 
studied, it was necessary to condense the readers’ reactions into man- 
ageable form. The vague impressions that this investigator received 
from reading several hundred pages of criticisms had to be reduced 
to concrete figures. The condensation of criticisms was performed 
by expressing the interest of pamphlets in one index and by classify- 
ing the thousands of criticisms under a few heads. The scale was 
called the Pamphlet Preference Index j the classification of reasons 
was reduced from thirty-eight criteria which have been discussed 
before to the fourteen main standards in the Combined List. 

A Pamphlet Preference Index, which gave the order of readers’ 
preferences for the twelve pamphlets, was compiled. The Index is 
an instrument for comparing known qualities of the pamphlets with 
the groups’ order of preference for them. 

Two different sources furnished the basis for the Pamphlet Pref- 
erence Index. The first was the readers’ general rating of the pam- 
phlets on a three-point scale. The second was the judgment which 
they expressed by specific ratings of each section of the pamphlets 
on a five-point scale. Following his reading of the pamphlet, as we 
have said before, the reader marked it “very interesting” (i), “inter- 
esting” (2), or “dull” (3). Each vote was given a weighting of i, 
2, or 3. The weighted votes were added together, and the sum 
was divided by the number of people voting. The average ratings 

spected for its probable bearing upon the readers’ opinions, but most of it revealed 
no clear-cut distinctions that could be used to show the correctness of the hypotheses 
leading to the request. 
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thus obtained for each pamphlet were ranked from most interesting 
to least interesting. This yielded ranks for general preference. 

Ranks for section preference were next obtained. Every section in 
each of the pamphlets was rated on a five-point scale: “very interest- 
ing” (i), “rather interesting” (2), “interesting” (3), “slightly inter- 
esting” (4), and “dull” (5). The average preference for each sec- 
tion was obtained from the weighted sum of the ratings in the same 
manner as the average preference for pamphlets. The average of 
all the sections in a pamphlet were added, and the sum was divided 
by the number of sections. The resulting dividends for the twelve 
pamphlets were ranked from most to least interesting, showing the 
order of preference for pamphlets according to their section ratings. 

These ranks on the basis of preference for sections of the pam- 
phlets and of preference for pamphlets as a whole were combined, as 
shown in Table VI, to give the Pamphlet Preference Index. 

Table VI 


The Pamphlet Preference Index: Combined Ranks of Twelve Pamphlets 
on the Basis of Preference for Pamphlet as a Whole and by Sections 


Pamphlet 

Rank on 
Pamphlet 
Preference 

Rank on 
Section 
Preference 

Sum of 
Section 
and 

Pamphlet 

Ranks 

Combined 

Rank 

Unemployment 

B 

9 

9 

18 

9 

C 

4 

4 

8 

3-5 

D 

I 

2 

3 

1-5 

War and Peace 

E 

.. 10.5 

II 

21.5 

II 

F 

•• 3 

7 

10 

5 

G 

a 

I 

3 

1-5 

Government 

H 

• 7-5 

10 

17.5 

8 

I 

6 

6 

12 

6 

K 

• 7-5 

5 

12. s 

7 

Unstable Money 

0 

10.5 

12 

22.5 

12 

P 

• 5 

3 

8 

3'5 

Q 

12 

8 

20 

10 


Note. I always signifies most popular, and 12 least. The averages upon which these 
rankings are based are given in Appendix III A. 
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A brief examination of Table VI shows that the two most popular 
pamphlets, both in the combined rankings and in their constituents, 
are in the well liked series on peace and unemployment. Two of 
the least popular pamphlets are in the least liked series on money. 
Undoubtedly the general topic contnbutes to attitudes toward spe- 
cific pamphlets. But a well liked topic does not guarantee popu- 
larity: E, in the popular series on peace, was one of the poorly rated 
pamphlets. Conversely P was above average in preference, although 
it dealt with the little appreciated topic of money. 

Usually the section rank corresponded closely with preference 
for the whole pamphlet. The reason for the unfavorable section 
ratings on F and subsequent higher general preference, or for the 
good section ranks and low general impression of pamphlet Q, was 
not determined. 

An informal check on the accuracy of the Pamphlet Preference 
Index was obtained through another measure. This measure was 
the average ranking of interest in the three pamphlets performed by 
the readers when they finished each series. It was found that their 
average ranking corresponded exactly to the rankings of each series 
of pamphlets in the Pamphlet Preference Index. 

Most Frequently Mentioned Standards 

The thirty-eight criteria, under which all of the criticisms of pam- 
phlets were classified, were too cumbersome to be submitted to rank 
order correlation.® Therefore the thirty-eight criteria were grouped 
under the fourteen topics which were preceded by Arabic numerals 
in the Combined List in Table V. The frequency counts of each of 
the sub-heads were allocated to the main topics under which they 
occur. For instance, a, b, and c were classed with A 3 as part of the 
selection and quality of information. 

The fourteen chief standards which forty-five readers, in contact 
with concrete material, said affected their attitude toward the ma- 
terial are given in Table VII. The topics are arranged from most 
to least often mentioned. 

In studying the significance of the order of criteria derived from 
the entire group’s two chief modes of criticism, no one will be sur- 
prised to see the topic, literary quality, which includes a sub-topic 

The st.itistical method of discovering- the degree of relationship existing between 
different factors which is employed throughout this study is rank order correlation. 
The symbol of the correlation is the Greek letter, rho. For explanation of the 
method, see [2], p. 190. 
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Table VII 

Order of Importance of Fourteen Principal Criteria, Based on Com- 
bined Frequency of All General Criticisms and Section Reactions 


Rank Criteiion 


a Difficulty 

3 Selection and quality of information 

4 Literary techniques 

5 Techniques of controversy 

6 Summary and appeal at conclusion 

7 Emotional appeal 

8 Techniques of gaining reader interest 

9 Presentation of figures 

10 . . . . Plan and appeal of introduction 

11 Author’s expression of viewpoint 

12 Format 

13 Historical allusions 

14 Use of famous author’s name 


of vitality, together with difficulty at the top of the column in Table 
VII. Writers and critics of the “sugar coating school” will find it 
hard to explain why a demand for well selected and adequately 
treated information should rank third, or why literary techniques, 
which stress the need of clear explanations, should follow in fourth 
place. Since the experimental material dealt with current contro- 
versial subjects, it is to be expected that the methods used by authors 
under techniques of controversy would receive frequent comment 
from readers. 

The topics in the lower half of the list received fewer comments, 
partly because they treated specialized aspects which did not occur as 
frequently as did the need, say, of adequate information. For in- 
stance, the quantity of historical data in the twelve pamphlets was 
neither large nor obtrusive and did not receive much attention from 
readers. Frequency of opportunity for comment is only a partial 
explanation for variation in rankings: there were twelve introduc- 
tions and twelve conclusions, yet the conclusions induced many 
more specific comments than the' introductions. The list as pre- 
sented gives an interesting picture of the averaged judgments of this 
group of experimental readers. 
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VariaUlUy of the Grouf 

A skeptical reader might well respond to the list of criteria just 
presented with, “Interesting, if true.” Several methods were em- 
ployed to find out how reliable the ranking of criteria in their order 
of importance could be considered. The order of criteria presented 
in Table VII was a composite of general and of section criticisms. 
Two separate steps are required to discover whether the ranks would 
generally occur in the stated order. First, the variability of the group 
m making its criticisms by a single method should be analyzed. 
Second, the amount of variability produced by two different methods 
of determining criteria should be discovered. 

A single pamphlet was selected for analysis. Pamphlet I, Modern 
Economic Systems, was chosen because it had demonstrated a popu- 
larity above the average, and had elicited thoughtful, unemotional 
responses. Every section reaction made by readers in commenting on 
pamphlet I was classified. The papers containing each reader’s 
criticisms were then sorted in two piles at random, and the com- 
ments for each pile were tabulated under their proper classifications. 
The rank order of frequency of the fourteen criteria for one random 
sub-group was correlated with that for the other random sub-group. 
The correlation between section reactions of random halves of the 
whole group was .93. 

When halves of the group were compared in the same way on the 
rank order of frequency of classified general criticisms, the correla- 
tion was .89, These high correlations indicate that the rank order 
of topics obtained by either method was reliable for this group of 
forty-five readers. 

Criticisms made by the whole group were kept together when the 
different methods of criticizing pamphlets were compared. The rank 
order of all the classified general criticisms on pamphlet 1 was cor- 
related with the rank order of section reactions. Rho was naturally 
lower than when self-correlations of single methods of criticism were 
made; the correlation was .77.® 

When all twelve pamphlets were considered, the rank order cor- 
relation between general and section criticisms was .68.® The correla- 
tion of .68 is statistically significant; rankings made by general 
criticisms and by section reactions frequently are almost the same. 
The two methods do, however, elicit sufficient differences in empha- 
sis on standards to warrant the use of both techniques. The ranks 
which have just been discussed, showing the relative importance 

” The computation of these correlations is given in Appendix III C. 
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of the standards in section reactions and in general criticisms, are 

given in Graph 2. 


Section renetlons 
Criterion RanJt 


General erltlciBma 
Bank Criterion 


Literary techniques 
Literary quality 
Difficulty 

Quality of Information 
Techniques of controversy 
Conclusion 
Introduction 

Relating: subject to reader 
Emotional appeal 
Presentation of figures 
Author's viewpoint 
Historical allusions 
Format 

Use of famous author 



Literary quality 
Dlfflpulty 

Quality of Information 
Techniques of controversy 
Emotional appeal 
Presentation of tlgvrea 
Author's viewpoint 
Relating subject to reader 
Conclusion 
ilterary techniques 
Format 

Introduction 
Historical allusions 
Use of famous author 


Bho la .eS 

Graph *. Criteria Ranks of Section Reactions Contiasted with Criteria Ranks of 
General Criticisms for Twelve Pamphlets 

The most striking feature of Graph 2 is the comparatively close 
agreement between rankings for the most important criteria (the four 
highest) and for the least important (the three lowest). Most of the 
variation, with the exception of comments on literary techniques, 
was found in the central six topics. The need of clear explanations 
and the remarks about dramatic techniques, combined under literary 
techniques, held chief place in the section analysis of pamphlets; 
they were mentioned infrequently in the final criticisms. 

Graph 2 indicates that both a detailed examination and a general 
criticism of the pamphlets on controversial problems led readers to 
judge them largely according to their literary quality, their difii- 
culty, the quality and selection of information, and the techniques of 
controversy which the author used. If section reactions were the 
instrument, introductions and conclusions received more attention 
than they did in general criticisms. The general criticisms stressed 
the emotional appeal of the literature, its presentation of figures, and 
the author’s display of viewpoint more than the detailed analysis did. 
According to both methods, format, historical presentations, and use 
of a famous author were least important. 

The variability of the group in mentioning the various standards 
has just been presented. The experimental group’s variation in its 
preference for the pamphlets will now be discussed. Knowing the 
order in which the fourteen major criteria were ranked was of gen- 
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eral interest to other investigators, both for guidance in research 
work and as an item of information. Other investigators are inter- 
ested in the experimental readers’ relative preference for twelve 
pamphlets only in order to interpret the responses of the group. 
It makes no difeence to them whether the order of preference estab- 
lished here is universal, or even whether the same group could be 
predicted to rank the pamphlets in the same order again. An indica- 
tion is necessary, though, of the internal consistency of group choices, 
in order to derive generalizations about the experimental group and 
its attitudes. 

The question to be answered is. How widely would random 
portions of the group differ in their indications of preference? Dis- 
covery of the random expectancy of preference rankings will tell 
whether discrepancies in choices between selected sub-groups are 
accidental, or whether they might rather be due to the infl^uence of a 
selective factor. 

The most accurate designation of preferences for pamphlets was 
made by the experimental subjects when they ranked three pamphlets 
on the same topic in order of preference. The strong and weak points 
of each pamphlet had to be weighed before this ranking was per- 
formed. Since the readers never ranked all twelve pamphlets 
together, the three rpkings in each series of four were not directly 
comparable to one another. None of the usual statistical methods of 
correlation showed the relationship between expressions of preference 
for all twelve pamphlets. 

A simple statistical device was used to compare the differences 
between two groups’ placement of each pamphlet directly. The 
average rating of a pamphlet by one random sub-group was sub- 
tracted from the average pamphlet rating of the other sub-group, and 
the difference was squared. The sum of the twelve squared differ- 
ences gave an indication of the amount of divergence between two 
groups’ opinions of all the pamphlets. 

This may be illustrated with pamphlet B5 the figures are given in 
Table VIII. Each of the members of random Group A rated pam- 
phlet B either i, 2, or 3 m interest. The average of the ratings per- 
formed by the twenty-two readers in Group A was 2.23. The aver- 
age rating upon Pamphlet B by Group B was 2.32. The difference 
between the averages of Groups A and B was .09; the squared dif- 
ference was .0081. The two groups agreed rather closely upon 
their rating of pamphlet B. , 

The average preferences of random halves of the group can be 
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compared from data in Table VIII. A clear difference between the 
first and second and third place ratings of the pamphlets in each 
series was generally distinguished. Groups A and B differed little 
from one another in their ratings of the series dealing with unem- 
ployment and government. In the series of pamphlets dealing with 
war, random Group A showed little choice between the pamphlets j 
but random Group B indicated a strong preference for pamphlet F, 
and a strong dislike of E. While Group B did not like O, in the 
money series, Group A ranked O as average. 


Table VIII 

Average Rankings for Preference of Each Pamphlet in Its Series of 
Three,* as Indicated by Random Groups 


Pamphlet 

Average 
Ranking by 
Group At 
(I) 

Average 
Ranking by 
Group Bf 
( 4 ) 

Difference 

between 

Ranks 

(3) 

Squared 

Difference 

(4) 

Unemployment 

B 

.. 2.23 

2.32 

- 09 

.0081 

C 

a. 09 

2.05 

.04 

.0016 

D . . 

. 1. 41 

1.58 

-•17 

.0289 

Peace 

E 

1.79 

2.33 

-•54 

.2916 

F 

2.00 

1-55 

•45 

.2025 

G 

2.00 

2.00 

.00 

.0000 

Government 

H 

. 2 . 14 

2.15 

— 01 

.0001 

I 

1.76 

1.84 

- 08 

.0064 

K 

• t -99 

1.83 

16 

. 02(6 

Money 

0 

2,10 

2.50 

- 40 

. 1600 

P 

1 59 

1.83 

- 24 

.0576 

Q 

. 2.15 

I 64 

• 51 

.2601 

Sum of squared difference 



■ 1-043 


Note. 1,00 would signify a rating of most popular by all readers 
* The readers ranked each three pamphlets in a series in order of preference, 
•f Groups A and B are random hdves of the group of 44 readers. 


The third column in Table VIII, showing the difference between 
the average preference rankings of the two groups, is Interesting. 
There was a difference of less than .1 between the two groups’ aver- 
ages for five pamphlets, and a comparatively great difference of .4 or 
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.5 for four pamphlets. The variability in preferences of random 
halves of the group indicates variability in the group as a whole. 
The sum of the last column, 1.043, is a convenient condensation. In 
order to ascertain the amount of variability in twelve choices by 
Groups A and B, the figure must be compared with other sums of 
the squared differences of averages expressed by selected groups. 
(See Table IX B, page loi.) 

There are two impressions which need to be transferred from this 
discussion of random groups to the next section on selected groups. 
First, when the section reactions of random Groups A and B were 
compared for ranking of criteria, a correlation of .93 was obtained. 
Second, a fairly wide difference in preference for pamphlets between 
random Groups A and B was indicated by the sum of the squared 
difference, 1.043. 

Similmties and Differences among Selected Groufs 

The heart of one of the problems of providing appropriate reading 
materials for large numbers of people lies in the analysis of small, 
selected groups. It is obvious that the “buckshot method” of sending 
a broadside of print aimed at no one in particular, in the hope of 
hitting enough readers to pay for a book, has not been particularly 
effective from the viewpoint either of publisher-producers or of adult 
educator-consumers. The most fundamental remedy for this diffi- 
culty is the study of the groups who will use reading materials, for 
the purpose of obtaining books that will fit their needs. Not only 
should whole groups be analyzed, but individual needs should be 
met. Exacting care should be taken to fiU individual and small group 
desires as adequately as possible. 

Analysis of individuals’ reactions to reading materials is a long 
process. It would be undesirable as well as impossible to collect 
information on individual preferences to guide the selection of each 
piece of printed curriculum material. There is another solution to 
the problem. The saving factor lies in the common elements of 
people’s tastes. A book that one person likes to read should fit the 
needs of other people like him5 literature prepared for one group 
should be well suited to other similar groups. 

As fast as the analyzing process is carried on, a synthesizing, gen- 
eralizing process should be set in motion. In studying one group, 
factors influencing reading tastes which might also be operative in 
other groups are sought. Individuals are analyzed to discover their 
similarities to other people, as well as their differences from others. 
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When small groups have been compared- to find their similarities a 
number of times, and the same relationship continues to exist, then 
general rules can be formulated. 

For instance, in a moment we will compare the reactions of the 
men and the women in this experiment. It will be shown that in 
the reading of current problems literature their standards of judg- 
ment and their reading preferences are closely similar. If this simi- 
larity prevails on a number of tests, it would be possible to set up as 
a working hypothesis that, granted an initial interest of the same 
strength, men and women enjoy reading the same materials with no 
need for sex differentiation. On the other hand, a wide difference 
in preference for reading materials will be shown between better 
and less educated sub-groups. If further studies show that difference 
in number of years of schooling may have been the primary cause 
of this difference in reading tastes, then permanent provision should 
be made by editors to care for readers on different educational levels. 

This information can only be secured by observing the effect of 
specific reading material upon groups and individuals. All the groups 
and individuals in the country will not have to be studied, because 
every careful analysis of small groups will contribute to total knowl- 
edge so rapidly that fairly definite guides to preparation and selection 
of reading could be proposed before long. 

The relation of three facts about the experimental group to the 
judgments of criteria and the preferences for pamphlets expressed 
by them were examined. The first was a general fact, a comparison 
of men with women. The second was more specialized, the contrast 
of two levels of education. The third, of direct significance to this 
problem, was a comparison of a group that had had college training 
along the line of the subject matter of the pamphlets with another 
group that did not have such training. 

In order to discover the relation between these facts and the evalua- 
tion of literature, two questions were asked about these sub-groups. 
First, did the contrasted groups mention approximately the same 
reasons for liking or disliking the material? The answer to this 
question would determine to some extent whether different or similar 
standards might be applied in selecting reading matter for them. 
Second, how closely did the sub-groups agree in their preference for 
pamphlets? Disagreement upon a piece of literature is produced in 
large measure by differences in past experiences. If people with one 
experience in common (such as being feminine) disagree with a group 
having a different experience in common (that of being masculine). 
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then that experience may be judged to have some effect on prefer- 
ence. We are searching for experiences that may be predicted to 
influence attitudes toward literature. 

The answer to the first question is based upon an analysis of the 
responses to pamphlet I, which, as we have said before, drew rea- 
soned unemotional criticisms from readers. A high correlation be- 
tween standards of judgment was noted earlier: the section reactions 
of one random half of the group were compared with the section 
reactions of the other half. Let us see what happened when two 
sub-groups, studied because of their differences, presented their rank 
order of standards. 


-Men Women 

Section reactions Section reactions 

Criterion Rant Rank Criterion 


Literary techniques 
Quality of Information 
Difficulty 

Techniques of controversy 
LlteraiV quality 
Author's vlev/polnt 
Introduction 
Smotlonal appeal 
Conclusion 

Relating subject to reader 
Presentation of figures 
Hlstorlosl allusions 
Format 

Use of famous author 



Literary tec&ilques 
Difficulty 

Quality of Information 
Techniques of controversy 
Literary quality 
Presentation of figures 
Introduction 
Conclusion 
Author's viewpoint 
Historical allusions 
Emotional appeal 
Relating subject to reader 
Format 

Use of famous. author 


Rho Is .89 

Graph 3. Men Contrasted with Women on Their Order of Mentioned Criteria in 
Section Criticisms of Pamphlet I 

There were twelve women in the experiment. Their order of 
mentioning topics influencing judgment was compared with that of 
the remainder of the group, thirty-three men. The correlation was 
.89. The agreement in ranking was almost unanimous on the first 
five and on the last two criteria. The graph shows that quality of 
explanation, difficulty, selection of information, and techniques of 
controversy were most important for the women. The men com- 
mented upon selection of information somewhat oftener than they 
did difficulty. The largest difference between the sub-groups was in 
presentation of figures, which women cited as a reason for their pref- 
erence for or dislike of pamphlets five ranks oftener than men did. 

Thirteen students who had pursued courses in the social sciences 
were compared with fifteen students who either had had no such 
courses or who had had a single one in high school economics. A 1 
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though the correlation of .85 was lower than that for men contrasted 
with women. Graph 4 shows a striking agreement on most of the 
rankings. In nine out of fourteen instances the rankings by the two 
groups were identical. The first five ranks and the last two were in 
the same order as that upon which the men and women agreed. The 
reason for a lower correlation than between the ranks of the standards 
erected by men and women is plain in the lines at the center of Graph 
4, which are longer than those in Graph 3. 

Students Nonstudents 

Crlterlon”'^'°^^°” Rank ^ Criterion 



Literary techniques 
Difficulty 

Quality of Information 
Techniques of controversy 
Literary quality 
Introduction 
Conclusion 
Emotional appeal 
Author's viewpoint 
Historical allusions 
Presentation of figures 
Relating subject to reader 
Format 

Use of famous author 


Graph 4.. Social Science Students Contrasted with Non-Students of Social Science 
in Order of Mentioning Ciiteria in Section Criticisms of Pamphlet I 


One might expect students of the social sciences to be critical or 
reserved in their opinions} the contrary appeared to be true in the 
experimental group. They gave evidence of an emotional reaction 
five rankings above the group which had not studied social science} 
they criticized the author’s viewpoint three rankings more frequently 
than the other group did. The untrained students commented much 
oftener on the conclusion. 

In considering the sub-groups that differed in education, the eight 
postgraduates gave a total of iia criticisms, whereas the eight less 
voluble two-year students gave only 72 criticisms. A high correla- 
tion between the two groups for frequency of mention of standards 
was found, .89. However, this was partially due to the coincidence 
of rankings. There was, nevertheless, the same tendency for these 
two sub-groups to place standards in about the same order as the other 
sub-groups did. The one exception is that the two-year college group 
ranked literary quality second and presentation of figures fifth, re- 
versing the order followed by the other five sub-groups. 
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TECHNIQUES OF THE INVESTIGATION 

A close agreement has just been presented between the criteria that 
divergent groups named when they evaluated pamphlets. Less 
unanimity is found in their preference for pamphlets. Although they 
mentioned almost the same standards in judging the material, they 
did not necessarily have the same preferences for the material. The 
reason for this seeming incongruity is that one sub-group may have 
mentioned positive criticisms under the heading of a standard and 
the other sub-group may have mentioned negative criticisms. Qual- 
ity of explanation remains a dominant heading, whether the reader 
said that the explanations were good or poor. In order to find out 
why the postgraduate group thought one pamphlet excellent while 
the undergraduate group considered it poor, it would be necessary 
to inspect the comments by each group under the various standards.’^ 

In the discussion of the preferences of random groups,® a method 
was shown whereby the average ratings on a pamphlet by two groups 
might be contrasted. The difference between two groups’ ratings of 
each pamphlet was obtained, squared, and added. The sura of the 
squared differences was 1.043 for random halves of the experimental 
group. 

The average differences between preference ratings of men and 
women were far less than those between the random samples. A 
squared difference sum of .5473 indicated that the women agreed with 
the men on the degree of interest of most of the pamphlets. The 
disagreement between the social science and non-social science groups 
was little more than random, with a squared difference sum of 1.200. 
The real disagreement occurred between those who had been to 
college two years or less and the other sub-group with postgraduate 
degrees. For them, the squared difference sum of interest ratings 
was 3.525. Table IX shows these differences in more detail. 

An examination of Table IX, part A, reveals the agreement and 
divergence of opinion for each of the sub-groups in their ratings of 
pamphlets. The averages of random sub-groups, when compared 
with other averages, seemed to be really random. The random sub- 
groups’ agreement on H was inconsistent with disagreements about 
H by the other sub-groups. The random sub-groups dissented on 
O and Q, while the other sub-groups did not. The men and women 
agreed closely on all the pamphlets but E and Q. The social science 
and non-social science students disagreed by more than three tenths 
of a point upon only B, H, and K. The sub-groups which were 

^This inspection is made in Chap V, pp. 140-42. 

“See Chap. IV, pp. 92-95. 
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Table IX 


Comparison of Sub-Groups with Varied Backgrounds in Respect to 
Average Preference for Pamphlets, and to Mention of Criteria 


A. Difference* between contrasted sub-groups' average ranking of each 
pamphlet m its series of three 




Social 

2 Years 


r ! I 

Random 

Men, 

Science, 

College, 

Sum of 

i aiiip ilei 

H n 1 vp.<t 

Women 

Non- 

Post- 

Difference 




Science 

graduate 



(45) 

(45) 

(28) 

(16) 


Unemployment 





t 

B 

. .09 

.09 

36 

.50 

•95 

C 

. .04 

.12 

.21 

.04 

•37 

D 

•17 

.04 

.07 

.42 

•53 

Peace 






E 

•S 4 

•54 

.21 

.88 

1.63 

F 

.45 

•17 

•23 

.29 

.69 

G 

.00 

.08 

•17 

•63 

.88 

Government 






H 

.01 

.16 

•75 

1.05 

1.96 

I 

. .08 

.00 

.00 

.29 

•29 

K 

.16 

17 

.56 

.76 

1.49 

Money 






0 

.40 

.12 

•13 

.07 

•32 

P 

.24 

.07 

.00 

•23 

•30 

Q 

• 51 

•35 

.05 

14 

54 


* Method of deriving difference is illustrated in Table VIII. 

t This column contains sum of difference of selected groups m second, third, and fourth 
columns. 


compared on the basis of education disagreed widely on half of the 
pamphlets j they reached approximately the same conclusion upon 
only C and O. 

The last column in Table IX, part A, telescopes the impression 
of the other columns to show which pamphlets produced wide and 
which little variation in opinions. There was least disagreement in 
rating pamphlet I for preferences, while H induced the widest varia- 
tion among sub-groups. 

In part B of Table IX, the sura of the squared difference between 
each pair of contrasted groups in part A is given. This is a summary 
of information discussed in connection with part A. The correla- 
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Table IX {Continued) 


B. Correlation between sub-groups for importance of criteria; difference 
between pamphlet preferences 

Contrasted Groups 

CorrelationJ of 
Ranked Critcua 

Pamphlet Preference 
Squared Difference § 

Random halves {11 with 22) 

•93 

1-043 

Men (33) 

Women (12) 

.89 

S47 

College 2 or less yeai's (8) 

College 5 or more years (8) 

.89 

3-525 

3 courses in social sciences (15) 

I or no courses in social sciences (13) 

.85 

I .200 


t These data are based upon a comparison of 14 major criteria, as derived from sec- 
tion comments on PamplUet I. 

§ Explanation of measure is given in Chap. IV, p. 93. 


tions between criteria have also been discussed before.® Both columns 
of information are collected in part B for easy reference. 

The crucial question arising out of an examination of Table IX is: 
Why do the wide differences between choices of pamphlets occur.? 
Are they due to controversial factors within the pamphlets? Are 
they due to varying difficulty, to methods of presenting material, 
or to other qualities? Are they perhaps bi'ought about by differences 
in the past experience and ability of the readers? 

It is impossible to carry analysis of the factors which are related 
to preferences of the sub-groups further until more is known about 
the qualities of the pamphlets. Methods of analyzing qualities and 
the knowledge assembled from applying these methods to the twelve 
experimental pamphlets will be presented in Chapter V. Then the 
discussion of similarities and differences between the sub-groups con- 
trasted here will be resumed. 

The comparison of six types of sub-groups has shown that there is 
not much difference between the order of their frequency of citing 
specific reasons for judging materials. There was little, more than 
random difference between their preference for pamphlets, except 
between the opinions of subjects with less and more education. 

The one wide' difference found should not lead to the hasty con- 
clusion that the three or more years of schooling separating one group 

“See Chap. IV, pp. 91, 97-98. 
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from the other is the direct cause of the disagreement in pamphlet 
preference. The members with advanced degrees may have devel- 
oped a peculiar psychological slant on life, due to their unemployment 
in spite of superior training. The less schooled sub-group may have 
obtained experiences in their post-college days that the other group 
did not have. Neither small group may be representative of under- 
graduate and postgraduate students at large. 

EXPERIMENTAL CONDITIONS WHICH MIGHT AFFECT OPINIONS 

Among the conditions in this experiment which might have some 
influence upon the readers’ estimates of pamphlets were differences 
among the readers and in the nature of the experimental set-up. 
The previous section tried to isolate the effect of such elements as 
differing sex, training, or education upon the evaluations of the read- 
ers. Of equal importance to the interpretation of criticisms is the 
isolation of any factors within the experimental conditions which 
might have affected judgments. The presence of peculiar experimental 
conditions which bring peculiar results might invalidate the findings, 
unless such were taken into account in the interpretation of the data. 

The three experimental factors which were studied, in order to find 
the extent of their influence upon opinions, were the varying degree 
of interest of the readers in each of the subjects with which the 
pamphlets dealt, the order of reading three pamphlets compared 
with the readers’ evaluation of them, and the seriousness and thor- 
oughness with which readers carried out the assigned tasks. 

The Varying Tofics of Pamfhlets 

The subject of the pamphlets was undoubtedly among the strong- 
est determinants of attitudes toward the literature. This factor was 
taken into consideration before the experiment began by asking each 
reader to rank the topics under which the experimental pamphlets 
were grouped. The reader was told to place the four topics in the 
order in which he would normally be interested in reading about 
them, with i as most and 4 as least interesting. After reading all 
the pamphlets, each person was asked to rank the four topics again, 
this time in the order in which he had enjoyed reading about them. 
The rankings made before reading were averaged, and so were the 
rankings made after reading the pamphlets. The averaged ranks 
follow, in Table X. 

According to Table X, most of the group, whose members had 
little money about which to worry, were definitely uninterested in 
reading about unstable money before the study. This lack of interest 
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Table X 

Interest of Readers in the Experimental Topics 


Topic 

Average Rank* 
Before Reading 

Average Rank* 
After Reading 

Unemployment 

2.30 

2.23 

War and Peace 

2 24 

2.29 

Changing Governments . . 

2.34 

2-37 

Unstable Money . 

3-09 

3-09 


* a 23 is best liked, 3.09 is least liked. 


occurred in spite of the facts that four-fifths of the groups said they 
had taken courses in economics and that the newspapers contained 
a great deal of discussion about inflation at the time of the experi- 
ment. The supposedly well chosen pamphlets did not raise the 
average interest at all at the end. 

The averages in Table X show that the only definite choice among 
the topics was evinced in the distaste for literature dealing with 
money. The fluctuations of the other averages were not great 
enough to be significant, but the relative rankings among the three 
topics may be considered. Before the experiment began, the members 
were not quite as much interested in reading about their immediate 
problem of unemployment as about war and peace. After exposure 
to some interesting literature, they promoted their own problem to 
first choice; The dramatic issues involved in the discussion of war 
and peace were somewhat more appealing to them than was the 
topic, changing governments, both before and after the experiment. 
Indeed, the latter topic was given a lower third ranking after the 
readers’ contact with the three pamphlets on the subject. 

The element of varying interest in the different topics was held 
constant to a large extent, since three pamphlets on the same subject 
were compared only with one another. However, interest in the 
topic may have operated in a subtler way to determine which of the 
three pamphlets was best liked. If a reader were familiar with a 
topic, he might choose one pamphlet j if the topic were new to him, 
he might prefer another. 

This hypothesis was proved in the series on unstable money. The 
people who were really interested in the topic of money (who checked 
the topic as interesting and who ranked it one) liked pamphlet Q 
very much. Q gave a simple, full, clear explanation of the meaning 
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o£ inflation. Another group that did not want to read about money 
(who did not check it, and who rated it four) preferred P, which 
was an amusing and well written defense of the author’s position on 
stable money. This difference in choice of pamphlets occasioned by 
a difference in interest in the subject was not found in any other series. 
The Order of Reading Pamfhlets 

The order in which three pamphlets in a series were read might 
affect the order of preference in either of two ways. The reader 
might become tired of the subject and tend to like the last pamphlet 
least. Or the reader’s interest might increase as he learned more 
about the topic, in which case he would rate the last pamphlet as 
preferable. If the factor of order of reading were too strong, that is 
to say, if the third pamphlet read was almost always rated least 
interesting, or most interesting, and the first pamphlet the reverse, 
the probability would be that the order of reading itself had too great 
an influence upon the rankings.^® 

The order of reading pamphlets and the ranking of the readers’ 
preference for each series were tabulated together. If the supposition 
of the last paragraph were correct, a disproportionately large per 
cent of ranking and reading orders would coincide. Actually, only 
26% of the ratings and order of reading coincided) 33% of the 
pamphlets read second or third were rated higher, that is, first or 
second) 31% of the pamphlets read first or second ranked lower. 
The distribution of ratings and reading order among coincidence, 
positive change, and negative change was random. The order of 
reading had no apparent influence upon ranking for preference. 
Care with Which Instructions Were Followed 

The last factor considered among those experimental conditions 
that might influence readers’ judgment was the care with which the 
reading and evaluation of the pamphlets was performed by the ex- 
perimental readers. The method used in studying this factor was 
the comparison of one conspicuously conscientious sub-group with 
another sub-group which was thought to be careless. 

The criticisms of four pamphlets, C, G, I, and P, one from each 
series, were selected for the survey. The “careful” papers for each 
were separated from the others by the assistant who was most 
familiar with them. Those were considered carefully written which 
presented sensible reasons for the ratings, in which the ratings were 

“ The element of fatigue would not influence the group’s ratings between series, 
since the series were read m random order. 
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spread somewhat over the five-point scale, and for which the instruc- 
tions were carefully followed. Half or more of the papers of each 
pamphlet fell under the careful heading. 

Ratings and criticisms of the seventeen people who were most 
careful, i.e., whose papers were selected for all four pamphlets, were 
compared with responses of fourteen who had no papers selected. 

The standards used in judging reading materials were nearly 
enough the same to give a correlation of .83 between the frequency 
of criteria for the “careful” and the “careless” sub-groups. The three 
criteria on which they differed most widely give some clue to the 
psychology of the “careless” group. The careful readers made more 
comments about the method of introducing the topic j the “careless” 
mentioned a great deal more often the way facts were presented, and 
somewhat more often the emotional appeal of the pamphlet, 

A comparison of the section ratings of the four pamphlets made 
by the two sub-groups revealed the fact that the “careless” group 
found the pamphlets far less interesting. In a compilation of the 
ratings of forty-nine sections, only seven, or one-seventh, were rated 
more interesting by the “careless” group than the median section in 
interest as judged by the careful group. These data mean that in 
the eyes of the “careless” sub-group, a very small proportion of 
sections were interesting; the careful sub-group considered at least 
half of the sections interesting. 

Closer examination of the “careless” papers showed that they fell 
into four classes. The first class was written by readers who thought 
the pamphlet too poor to read. As one man in this group expressed 
it, more dramatically but no more feelingly than the others: “Sec- 
tion I. Fit for a tabloid mind. Section 2, Very sad, very sad. 
Section 3. A little better, but not much.” His general estimate of 
the pamphlet was, “This type of article is fit only for person who 
must have information sugar-coated, who must get information by 
indirection. Gushy. Blah-human interest'” The critic was not fol- 
lowing directions, for he gives very little in the way of information 
about why he does not like the pamphlet. But he could not be called 
careless in the usual sense of the word. 

A second class was not sufficiently analytical to give reasons that 
have a value for the study. The members said: “Well presented.” 
“Proven facts.” “Person^ tinge.” “Not clear — ^too simple.” “De- 
moralizing aspect.” It is probable that these represent the type, 
found in any study group, who are unable to analyze and to intro- 
spect for deeper meanings. 
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A third class, of whom there were only two in the group of forty- 
five, could be definitely classified as incompetent. Each of them spent 
some time with us every day as we pointed out errors, made con- 
crete suggestions for improvement, and showed them examples of 
what we wanted. They could not comply with our directions, though 
they earnestly attempted to work. 

In the papers examined, there was only one which belonged to 
the fourth group, that of obvious unwiUmgness to follow instruc- 
tions. The responses may well have been written without glancing 
at the pamphlet. They were not consistent, for the section reactions 
contained such remarks as “Impressively written,” “Written in an 
understandable manner,” while on the back of the same paper the 
general criticism was “Dull. This must have been written for ele- 
mentary students.” 

The sub-group that was labeled careless, then, could not in gen- 
eral be rightfully called so. With the exception of a few cases, they 
might be considered a normal part of any group, including those 
who are not greatly interested and those who do not have the ability 
or the will to do the task at hand thoroughly. To remove the opin- 
ions of this minority would be to give undue weight to the most 
interested and intelligent members. Although the so-called “care- 
less” group did influence the results, incorporating their criticisms 
in the data made the group more nearly normal and did no harm. 

The less careful group, which was a minority of the whole group, 
contributed to the purposes of this study. They helped to secure 
reasons which clarified the criteria for selecting reading material. 
This group agreed closely enough with the “careful” group in fre- 
quency of mentioning criteria. This less mature group was essential 
for the determination of the spread of interest in the various pam- 
phlets. 


EXPERIMENTATION WITH A CHECK LIST 

The pamphlets were submitted to two other widely different 
groups. Each person in the two groups read only two pamphlets 
on a single topic. He checked a list of reasons for judging the 
pamphlets instead of writing out free responses. Handicapped by 
these limitations, the findings contributed to the field of techniques 
rather than to the verification of criteria. 

The long list of reasons for liking or disliking pamphlets obtained 
from the Civil Works Administration group was telescoped to a few 
salient points upon which it was desirable to have information. 
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The assistance of a class of Negro adults , graduating from an 
eighth grade evening school in Harlem had been proffered. Con- 
sultation with their principal and with others who had worjked with 
similar classes indicated that their literacy and comprehension level 
was not high. Therefore the check list was set up in the simplest 
manner that could be devised.^’- Directions were made very explicit, 
and each standard was phrased in non-technical words. For instance, 
A below was the original form and B the changed form: 

A. Body; Positive Reasons A. Body: Negative Reasons 

I. Topic is of current interest. I. Topic not timely. 

* * * * * 

B. Tilings I Like About the Main. B. Things I do not Like About the 

Part of This Book. Main Part of This Book. 

I. People and newspapers are talk- 1. This subject is too old; nobody 
mg about this subject now. talks about It now. 

Several experts gave their advice about the form of wording of the 
check list. Opinions were conflicting about the device of arranging 
positive and negative reasons parallel to one another on each side 
of the page. One professor suggested a single statement with -f-, 
0, — checked to indicate variations of opinion, but this seemed too 
difficult for people not accustomed to dealing with such symbols. 
Actually, few negatives were marked. The reason for this may have 
been due to a hesitancy to criticize, as well as to the form of the 
check list. 

In the short time at the reader’s disposal, only two pamphlets 
could be read. Therefore, one was eliminated from each series. 
The unemployment series lost the popular D because its hundred 
pages made it too long. F and K, which were removed from the 
war and the government series, had aroused emotional reactions that 
made rational analysis difficult. The hardest and least popular, O, 
was taken from unstable money. 

The check lists, with two pamphlets for each student, were taken 
to Harlem. There the general idea was explained to an earnest audi- 
ence. Each man was given his choice of any of the four series. 
They started work, reading the introduction of one pamphlet and 
checking the reasons relating to the introduction, while two investi- 
gators went around helping them, explaining individually the direc- 
tions which had already been given in writing and orally to the 
group. Almost half of the group of thirty-three needed assistance 
in understanding and checking the forms. The class members took 
the pamphlets home, and the completed check lists were returned 

“ See Appendix II C. 
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in five days. Twenty-three members read and checked both pam- 
phlets. The form which was given to the Harlem group contained 
no room for general criticisms. Some of the readers either wrote or 
made verbal comments which were so illuminating that general 
criticisms were incorporated in the experimental technique. When 
the check list was used again, the readers were asked to write their 
general criticism of the pamphlets. These comments disclosed their 
degree of comprehension of the material and also gave additional 
insight into their standards of judgment. 

A lively suburban group which had grown out of a recreation hour 
for men was given the material to read and the blanks to check. Their 
median educational level was two years of college. Fifteen people 
from this discussion group read two pamphlets each and checked 
the list without any verbal instructions or aid. They had no diffi- 
culty in following directions or in interpreting the check list. Indeed, 
it had apparently been over-simplified for them, for one man wrote, 
“I did not check any negatives because they seemed so silly.” This 
comment suggested that the test material as well as the reading must 
be specifically adapted to the group that is to use it. 

The chief purpose of submitting the check list and pamphlets to 
other groups was to discover which standards were of most im- 
portance to different kinds of groups. The frequency of mention 
of topics by the Civil Works Administration group, which had not 
been given a check list, was compared with that of Harlem and sub- 
urban groups. 

The comparison of criteria among the three groups was limited by 
two factors. First, the quality of format was over-emphasized in the 
check list. Therefore the Harlem and suburban groups made too 
frequent comments in comparison with the normal responses of the 
Civil Works Administration group. Second, the Harlem and sub- 
urban groups marked comparatively few negative reactions on the 
check list. Accordingly, the correlation between the Civil Works 
Administration and the other groups was restricted to positive stand- 
ards, exclusive of format. The Harlem and suburban groups were 
compared only on positive standards} the format was included in 
their correlation, since the two groups were subjected to the same 
research instrument. The degree of correlation between the various 
groups is shown in Table XI, part B. 

Part A in Table XI gives the data on the basis of which the nega- 
tive criticisms were omitted from the correlations. It is evident that 
abnormally few negative responses were made when the check list 
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Table XI 

Comparison Between Three Groups’ Judgment of Criteria 


[09 


A. Proportion of positive and negative judgments 


Group 

Insti ument 

Per Cent Responses Which Were: 
Positive Negative 

C. W. A 

.... Section reactions 

54% 

46% 

Harlem ... 

.... Check list 

89% 

11% 

Suburban . . , 

. . Check list 

90% 

10% 

Suburban . 

.... General comments 

74% 

a6% 


B. Correlations between positively stated criteria for three groups* 


Groups Compared 

Rho 

P. E. Rho 

C. W. A. with Harlem 


•13 

•13 

.05 

C. W. A. with Suburban 

. . 

Suburban with Harlem 

.' . ’ *81 


* Appendix III B contains the data from which the correlations are derived. 


was used, if the basis of comparison is the proportion of unfavorable 
criticisms made when readers were not restricted to a check list. Part 
B summarizes the correlation between the frequency of checking 
standards by the various groups. The Harlem and suburban groups 
attained a high correlation of agreement upon ranking of criteria, .81. 
The Harlem and Civil Works Administration groups agreed in their 
frequency of mentioning standards used in judging the same four 
pamphlets with the correlation of .39. The rank order correlation 
between the suburban and the Civil Works Administration groups 
was .40. These correlations, revealing some agreement in frequency 
of mentioning standards in spite of wide differences in types of the 
three groups and of their methods of evaluation, suggest that some 
of the same standards may influence all readers’ judgments. How- 
ever, the artificial exclusion of some of the data tends to invalidate 
the correlations. 


CRITICISMS OF THE TECHNIQUES 

During the progress of the experiment a choice frequently had 
to be made between two research techniques. When the experiment 
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was completed, there was reason to inquire whether all of the steps 
taken were necessary to obtain results, whether a fragment of the 
process was not almost as reliable as the whole process either for 
setting up standards or for obtaining reactions to definite materials. 
Since part of the object of this preliminary investigation of the cur- 
riculum is to question old methods and to develop new techniques 
which might be used by practical educators, some of these problems 
and their solution may be given to the reader. 

iVLethod of Classifying Standards 

The method of classifying the criteria was questioned several times 
during the experiment. The most thorough scientific technique is 
to request a number of judges to make the proper classification of each 
criticism. This method was believed to be unnecessary in an explora- 
tory study such as the present one; the approximate grouping of 
criticisms supplied the immediate demand for a defined list of stand- 
ards. An incomplete classification would act as a spur for further 
investigation, but would meanwhile be of some guidance to curricu- 
lum selection. 

The Combined List of Criteria formulated in this study is far 
from being a complete basis for further studies. The need of a scien- 
tifically approved classification which would provide at least the 
chief divisions under which could be fitted standards for judging any 
kind of adult non-fiction reading still remains. Till we have this 
classification, results will not be comparable from different investiga- 
tions upon different subjects. In order to build a scientific classifica- 
tion of criteria, investigators should obtain opinions of readers upon 
samples in all of the important branches of adult study. A number 
of people should classify the reasons obtained, under controlled con- 
ditions, thus accumulating a consensus of agreement upon categories 
and classifications.^^ A general guide list of criteria would thus be 
built which could be used for classifying all reasons given by readers 
or for eliciting opinions on any type of non-fiction literature. 

Relative Merits of General and Section Criticisms 

The relative merit of the general criticisms of whole pamphlets 
compared with the detailed criticism of each section was weighed. 
Section analyses call for a patience that may not be possessed by all 
of the volunteer groups who will probably be subjects for most later 
experiments. On the other hand, the general analysis may not indi- 

“ An exact method of classification was reported in [i], pp. 81-94. 



TECHNIQUES OF THE INVESTIGATION 


cate weak places in the material, or may not throw exceptionally 
strong passages into relief. Let us see what evidence may be brought 
to bear on the problem. 

The rank order correlation between the general and the section 
reactions for all of the pamphlets was .68.^® This indicates a more 
than haphazard relation between the two methods of ranking, and 
shows that the same criticisms occurred, in proportion, roughly as 
frequently when general criticisms were made as when detailed com- 
ments were given. 

This correlation may be interpreted in two ways, depending on the 
use which is to be made of the data. If an experimenter is merely 
interested in discovering the standards that his adult class most often 
uses in choosing its reading, then the gathering of a number of gen- 
eral criticisms will be almost as effective as the request for specific 
reactions and will save time. As a tool for collecting criteria and 
for showing their relative importance, general criticisms are valid. 

If, on the other hand, the experimenter wants to test the strong 
and weak points of particular material, he will find a detailed ap- 
praisal of the reading — page by page, or chapter by chapter — to be 
invaluable. He should not dispense with general criticisms, even if 
specific analyses are carefully worded. The general criticisms empha- 
size some qualities which were not apparent to the reader in his 
detailed analysis, and the sectional criticisms mention flaws or virtues 
which are neglected in a general appraisal. 

A related problem, raised often by the readers, was the difficulty 
and trouble of rating small sections of pamphlets. Specific com- 
plaints from the readers referred chiefly to their interruption of an 
idea in order to evaluate a section. In the future, divisions should 
be made only with a complete break in the thought at ends of sections 
or of chapters. Occasionally in material as brief as these pamphlets, 
the pause in thought does not occur until the end of the book, as 
attested by one man who wrote, “Its author has entranced me so 
that I have made very poor decisions about my section comments.” 
Use of a Check List 

The remaining problem to be discussed is one that will confront 
other investigators of reading materials. Should a check list be pro- 
vided upon which readers can indicate opinions? Several arguments 
favor a check list that contains all the main standards for judging 
reading material. A good check list would be based upon a classifica- 
tion of all the chief standards which a person would ordinarily erect 

“ See Chap. IV, p. 90 and Appendix III C. 
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in judging literature. Time would be saved if, instead of writing a 
full reason, he checked the classification head to which his criticism 
belonged. Only a few criticisms might occur to him without help. 

The arguments opposing the use of a check list are of several kinds. 
The readers may check indiscriminately, instead of analyzing their 
own opinions and then checking the phrase that best expresses them. 
Then, too, the phraseology of a check list may not fully imply reac- 
tions that would be obtained in free responses. Further, we ordi- 
narily think in specific terms, whereas a check list is generalized. 
The Harlemite who liked pamphlet P because it told about cutting 
off babies’ heads in Japan (the passage has not been located) did not 
check the general reason concerning effective stories and examples. 
In like manner, most of the comments that one thinks of are specific 
and must be interpreted for classification in the generalized form. 
But it may be too much to ask some groups to translate their specific 
objections into generalized concepts. 

Finally, there are the difficulties of setting up an adequate form. 
The check list must be expressed in terms with which its users are 
acquainted. Some provision should be made for showing variations 
in the quality under consideration: plus, zero, minus, or rating on a 
positive-negative scale, or arrangement of positive and negative quali- 
ties in the parallel form utilized here. Then a decision must be 
reached concerning the extensiveness of the list. If it includes all 
possible alternatives, the users may be exhausted; and if it is ab- 
breviated, its completeness and often its clarity may be sacrificed. 

The most satisfactory compromise, we believe, is a guide to criti- 
cism such as that presented in Table II. It overcomes some of the 
objections raised to the check list, but incorporates its advantages. 

The recommended guide is based upon all of the main standards 
of evaluation found in this experiment. If the guide requires readers 
to write responses, it is difficult for them to escape making a discrimi- 
nating analysis. The guide presents an opportunity for citing specific 
criticisms under general topics. It forces readers to express positive 
or negative opinions in the course of evaluating each topic. The check 
may be as extensive or as short as the expressive ability of the judge. 
Finally, in case any points were omitted from the guide, the reader 
may write them in his general criticism. While the form presented 
in Chapter III requires more analysis of the returns by the investi- 
gator than a simple check list would, the readers would have made so 
many more significant responses that the extra work by the experi- 
menter would be amply justified. 
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SUMMARY 

After reading a number of books on adult education and talking 
to professional writers, publishers, and educators, it was apparent to 
the writer that no well defined methods for determining the reading 
matter that would be successful with adult study groups were avail- 
able. No experimental work was recorded in which ordinaiy readers 
made a detailed analysis of the standards that guided them in their 
attitudes toward reading. Accordingly, an experiment of this nature 
was made. 

A group of adults who were assembled for experimental purposes 
read pamphlets upon current controversial topics and wrote reasons 
for liking or disliking the literature. In order to discover the criteria 
used by the readers as they evaluated the literature, their criticisms 
were classified. Thirty-eight topics, under fourteen major heads, 
were organized from the criticisms into the Combined List of Criteria. 
The Combined List was compiled from four sources: the opinions 
of experts regarding standards which are used in selecting literature, 
the general criticisms of pamphlets by the experimental readers, the 
detailed section reactions and the reasons for snap judgment of 
pamphlets, both of which were likewise written by the readers. It 
was refined to show the relative importance of the various criteria; 
the frequency with which each criterion was mentioned provided the 
basis for grouping it in one of four quarters of importance. 

In addition to discovering the standards used by readers in evalu- 
ating the pamphlets, some characteristics of the experimental group 
were analyzed. The analysis took the form, first, of studying the 
preferences of the whole group and the reasons that they gave for 
their preferences. Then the preferences and reasons of sub-groups 
were inspected. 

The relative interest that members expressed in the various pam- 
phlets was shown by a Pamphlet Preference Index, which was com- 
posed of averaged ratings of the pamphlets and of sections of pam- 
phlets. In order to simplify correlations, the criticisms of the material 
were listed under the fourteen major standards in their order of 
importance j the criticisms of the pamphlets in general were con- 
trasted with the criticisms of sections of the pamphlets. 

Certain important facts about the experimental group obtained 
from a questionnaire formed the basis for selecting sub-groups to 
study the respects in which they were similar to or different from 
one another. Men’s tastes were so much like women’s that very little 
difference was seen in either their ranking of criteria for importance 
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or their ranking of pamphlets for preference. Likewise, the mem- 
bers who had taken social science courses akin to the material of the 
pamphlets showed no more than random differences on either score 
from members who had not pursued such courses. While the eight 
members who had been to college for two years or less agreed in 
their ranking of criteria with the eight who possessed postgraduate 
degrees, they disagreed widely on pamphlet preference. 

Three factors inherent in the experimental set-up, the interest in 
contents of pamphlets, the order of reading, and the care with which 
instructions were followed, wei*e examined for their influence upon 
results. The first factor to be studied was the topics about which 
pamphlets were written. The topics were found to produce a dif- 
ference in interest which must be taken into account in examining 
preferences for single pamphlets. Second, the order in which pam- 
phlets were read had no observable influence upon the order in which 
pamphlets were preferred. Third, when a group of “careful” read- 
ers was compared with a “careless” group, it was found that while 
the two sub-groups used approximately the same standards in judg- 
ing the pamphlets, the “careful” group liked the four pamphlets 
that were analyzed much better than the “careless” group did. No 
ground was found for assuming that almost all of the group was not 
making a conscientious appraisal of the experimental literature. 

Some of the techniques developed in this study were evaluated; 
the following suggestions for their improvement were made. First, 
while the list of criteria assembled here may be used as a guide to 
selection of curriculum materials, a list based upon other fields than 
current social problems should be prepared eventually for the use of 
leaders and research workers. Second, detailed as well as general 
criticisms of experimental literature should ordinarily be employed 
when the literature is being tested. Third, although the use of a 
check list or guide to reader criticisms presents some difficulty, an aid 
to criticisms such as that suggested in Chapter III should be used. 
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CHAPTER V 


THE EXPERIMENTAL READING MATERIALS 

Reactions to literature are based chiefly upon two factors: the char- 
acter of the reader and the nature of the literature. In Chapter IV 
the character of the experimental group was investigated. Differ- 
ences and similarities in the tastes of men and women, of college non- 
graduates and post-graduates, and of students trained and untrained 
in social sciences were pointed out. The next step is the analysis of 
the nature of the experimental pamphlets. There are a number of 
qualities within the pamphlets which may have influenced readers’ 
reactions. These qualities will be presented in order to indicate the 
form in which they existed in each pamphlet. The comparison of 
this objective information with expressed preferences of the experi- 
mental readers for the pamphlets will show the requirements for 
literature that were set by these readers. 

Eight qualities of the pamphlets were analyzed by from two to 
six assistants, who were selected for this purpose from the larger 
group of Civil Works Administration members. Then readers’ 
preferences and criticisms of the pamphlets were studied. The bodies 
of information about readers’ preferences and the quality of the 
reading matter were combined in a search for the criteria upon which 
preferences were based. The methods included in studying the pam- 
phlets and their effect on readers wiU be recounted in sufficient detail 
to permit other investigators to duplicate them. 

LABORATORY ANALYSIS OF PAMPHLETS 

Content 

One of the factors in a book that has most influence upon readers’ 
opinions undoubtedly is its content. What the author discusses is 
generally the single greatest factor producing interest or lack of it in 
the reader. Therefore, the starting point for analysis of any material 
is consideration of its content. 

Two or three assistants summarized the contents of each of the 
twelve pamphlets in short sketches of four to a dozen lines. The 
sketches were combined to form the basis for the presentation of 
”5 
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content that is given in Appendix III E and illustrated for two pam- 
phlets later in this chapter. 

Techniques of Relating Subject to Reader 

The opinions of readers may be influenced not only by what the 
author is talking about but by his style of telling it. A descriptive 
method was used in discovering those factors in the pamphlets that 
readers called “human interest.” With the following outline as a 
guide, each pamphlet was described by two or three assistants. 

1 . Sketch the content of the pamphlet briefly. 

2. What techniques does the author use to catch and hold reader interest? 

3. Does he seem to have one group of readers especially in mind as he writes? 

The interesting and varied devices which were used in the twelve 
experimental pamphlets are classified in Chapter VL^ The discov- 
ered techniques for relating a subject to the reader fit under heads of 
twelve of the fourteen chief criteria, showing that readers’ attention 
may be arrested by the skillful introduction of almost any of the 
qualities. Techniques obviously utilized to interest the reader were 
dialogue, dramatization, analogies, and pictures. Emotionalized 
language was employed, as was the opposite method of a scholarly 
style. A full presentation of facts, with adequate definition and ex- 
planation, was just as powerful in holding the interest of readers as 
were the more commonly conceived devices. The methods em- 
ployed by the author of each pamphlet are given in Appendix III E. 
A detailed statement of “human interest techniques” is made illus- 
tratively for pamphlets D and O in this chapter. 

Difficulty 

Since the method utilized by Gray and Leary [4] * to determine 
the difficulty of books for adults was not available when the pamphlet 
analysis was in process, the Winnetka formula [8] for determining 
the grade placement of books for children was used. With the aid 
of the Winnetka Chart for Determining Grade Placement of Chil- 
dren’s Books, printed by the Winnetka Public Schools, a sample of 
one thousand words from each pamphlet was tabulated. The first 
fifteen hundred words in difficulty (according to the Thorndike 
Word List [7] and certain other counts) were printed on the chart, 
with spaces for entering “uncommon” words alphabetically at the 

See Chap. VI, p. 193 and also Appendix III D. 

* Throughout this chapter numbers in brackets refer to bibliographical references 
to Chapter V, as given on page 159. 
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right. A sample of seventy-five sentences was classified as “simple” 
or “not simple” according to stated rules. 

All measures were combined in a formula which had as its elements 
(i) the total number of different words in a thousand, (2) the pro- 
portion of those words that are less common than the first fifteen 
hundred in frequency, (3) the ratio of simple to complex sentences, 
and (4) a statistical constant that had been found to aid in grade 
placement. The end product stated the school grade in which the 
book could be used to best advantage. Although the present study 
was not concerned with grade placement, the formula permitted the 
determination of the relative difficulty of books, and the chart was a 
convenient device for performing the first step in an analysis of diffi- 
culty, that of taking a sampling of words. 


Table XII 

Ranking of Pamphlets for Difficulty on the Basis of the 
Winnetka Formula 


Pamphlet 

Winnetka Formula 
Grade Placement 

Difficulty Index 
Rank 

Unemployment 

B 

9.4 

3 

C 

6.3 

11 

D 

.. . 8.2 

5 

Peace 

E 

.. . 6.6 

10 

F 

10 4 

I 

G 

81 

6 

Government 

H . .. . 

5-5 

12 

I 

78 

7 

K . . . 

8.5 

4 

Money 

0 . 

6.7 

9 

P . 

6 8 

8 

Q - 

9-7 

2 


Table XII states the grade placement of each of the pamphlets 
according to the Winnetka formula. It can be seen that none of the 
pamphlets is regarded as difficult reading: only one is difficult enough 
to rank in senior high school, and one, H, ranks in the fifth grade. 
Each series contains one pamphlet ranking in the most difficult third. 
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one in the central thirds and one in the easiest third. When the pam- 
phlets were chosen, there was some attempt to obtain a distribution of 
difficulty in each series, but the success of judging the distribution of 
difficulty among the series was unexpected. 

Difficulty of vocabulary and complexity of sentence structure do 
not necessarily constitute difficulty of reading materials, especially for 
adults. This conclusion is borne out by the fact that although Dale 
and Tyler [3] found that elements similar to those in the Winnetka 
formula — number of different non-technical words, number of hard 
non-technical words, and number of mdeterminant clauses — corre- 
lated with general tests of difficulty more closely than any other of 
several factors studied, the correlation which they obtained for these 
three combined factors was only .51. Gray and Leary [4, p. 138] 
found a somewhat higher correlation for difficulty scores and these 
five elements: the number of different hard words, the number of 
first, second, and third person pronouns, the average sentence length 
in words, the percentage of different words, and the number of 
prepositional phrases found in a sample passage of 100 words. Their 
correlation between a predicted average reading score and the five 
elements was .64. 

There are other factors involved in difficulty which have not been 
discovered by either formula. Such elements may be the clarity of 
sentence structure, or the skill with which sentences and paragraphs 
are woven into a unified sequence, or the limitation upon the number 
of concepts outside the range of the particular readers. These addi- 
tional elements may be impossible to isolate for purposes of scientific 
study} certainly they have not lent themselves to analysis to date. 

The probability that difficulty for mature readers includes impor- 
tant factors other than vocabulary is indicated by the fact that the 
readers in this experiment mentioned general difficulty of concepts, 
such as ‘‘subject well defined,” “too hard to read,” “non-technical,” 
“easy style,” etc., often enough to place that quality in the first quar- 
ter of important criteria.® Difficulty of terras, on the other hand, 
such as “good definition of words,” “do not understand many terms,” 
etc., was mentioned so rarely that it fell in the third quarter. An- 
other indication of a broader definition of difficulty is the failure of 
readers to rank the pamphlets for difficulty in the same order as the 
Difficulty Index (see page 117). 

For the present, however, the most practical determination of read- 
ing difficulty that can be made in advance of reader criticism is 
through the analysis of various mechanical aspects of the printed 

See Table V, p. 8z. 
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word with the Winnetka or the University of Chicago formula. The 
latter provides a reasonably accurate key to the difficulty of books 
for students of limited reading ability. 

Historical Allusions 

One of the factors that sometimes causes people to become biased 
for or against books is the presence of historical allusions. Historical 
references were described for each of the twelve pamphlets, and the 
comparative quantities were correlated with preferences. 

Four assistants followed these directions in analyzing the pam- 
phlets for historical allusions: 

Take a pamphlet, and list every section that contains historical data. “His- 
toiical” .applies to anything that occurred prior to the Roosevelt administration. 
(The study was made in April, 1934..) Classify the material according to 
the following outline: 

1. Is it used to provide a bachgrmmd for understanding the author’s posi- 
tion, or as an introduction? 

2. Is it used to illustrate a point? 

3. Is it used for comfmattve purposes, with several instances cited in refer- 
ence to one another? 

4. Other reasons. 

The most helpful analysis was made by an assistant who amplified 
the instructions by adding a short descriptive phrase after each classi- 
fication, as “Background — history of Muscle Shoals struggle.” 

Two assistants analyzed the quality of history in one pamphlet 
and three studied the remaining eleven. Each placed a check mark 
beside the number of every section containing history. The checks 
of the two or three workers coincided for seventy-four per cent of 
the sections. Seventy-six per cent of all sections checked by the 
workers as containing history were considered, after consistent and 
rigid examination by the author, to have historical allusions. 

The author’s opinion, influenced by that of the two or three 
assistants, was the basis for the preparation of a scale of quantity of 
history present in the twelve pamphlets. The number of lines of 
history in each section judged by the assistants to contain history 
were counted and added to find the total number of lines of history in 
the pamphlet. The proportion of lines with history to the total 
estimated lines in the pamphlets provided an index figure which could 
be ranked with the indexes of other pamphlets. The resulting scale 
runs from a pamphlet with ii lines of history, ranking first, to one 
with 495 lines, ranking twelfth. The ranks follow, in Table XIII. 
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Table XIII 

Ranking of Pamphlets by Proportion of Space Given to History 


Pamphlet 

Estimated 
Total Number 
of Lines 

Counted 

Proportion of 
Lines of History 
to Total 

Rank 

Unemployment 

B 

1 102 

42 

% 

3.8 

10 

C 

884 

17 

I 9 

II 

D 


195 

7-5 

8 

Peace 

E 

570 

168 

29 3 

2 

F 

1416 

11 

.8 

12 

G 

700 

68 

9-7 

6 

Government 

H 

560 

35 

6.4 

9 

I 

1020 

95 

9-3 

7 

K 

1122 

155 

13-8 

5 

Money 

.0 .. . . 

703 

126 

17.9 

3 

P 

768 

1 14 

14.8 

4 

Q 

• ■ 1363 

495 

36-2 

I 


There was little evident relationship between the length of the 
pamphlet and the amount of history included in it. The one with the 
fewest lines of history, F, was next to the longest. The longest pam- 
phlet, D, ranked eighth with a ratio of 7.5 lines of history. 

Some relationship between the content of the pamphlet and the 
amount of history was visible. Comparatively little historical back- 
ground was provided in the discussions of unemployment, since 
pamphlets B and C were chiefly concerned with steps taken under the 
Roosevelt administration, and D introduced history only for illustra- 
tion. Most of the historical evidence was introduced in the three 
discussions of inflation. In the series on peace there was more varia- 
tion: pamphlet F resorted to exhortation, but G introduced some his- 
torical data, and E was full of illustrations drawn from previous 
wars. 


Presentation of Figures 

The fact is scarcely debatable that the presence of figures, tables, 
or graphs in written material has an effect upon the attitude of many 
readers. Some declare that they ignore all tables, whatever their 
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import} others prefer concise listing of figures to generalized state- 
ments. Authors are aware of reader prejudices for or against pres- 
entation of statistical data and use various devices to overcome the 
prejudices. 

The amount of statistical data in each pamphlet was compared 
with reader preferences. The different methods used by authors in 
handling facts were described. The directions for discovering occur- 
rences and types of figures, which were given to the assistants, follow: 

List the section and page on which figures are found. Make a note of every 
occurrence of figures and written numbers except dates. State whether they 
are presented: 

I . In tables 

By graphs (see guide sheet) 

3. In running account, or context 

4. With dramatic or human application 

To insure uniformity in describing the graphs, a copy of a dozen 
common graphic forms with their labels was made. [6] Each assistant 
compared every graph with this standard copy. 

The expression, “In tables,” was used to include all figures in 
tabular or column style. “In running account” pertained to figures, 
either spelled or numerical, which were included in paragraphs in 
the context. The “dramatic application” of figures to common ex- 
perience was separated from the other contextual explanations in 
order to estimate whether it was as well liked a technique as its 
sponsors declare it to be. A sample of dramatic application is: 

. . . (The member bank of the Federal Reserve) does precisely what John 
Smith would do if, short of cash, he went to the bank and drew out $25 of 
his deposits in currency. ... If, when he goes to draw out his $23, John 
Smith is told that he has not that much on deposit, what does he do? ... 
If he is a business man with suificient standing, he borrows $100 on his 
note. ... It is precisely the same with the member bank. . . .”, etc. [2, 
P- 9] 

Such examples are common throughout these pamphlets. 

Either two or three assistants recorded each page that contained 
figures in the pamphlets. They agreed upon sixty-seven per cent of 
the instances of occurrence of figures. The author examined the pam- 
phlets in the light of the previous judgments, and ranked them ac- 
cording to the proportion of pages containing statistics. The ranks are 
given in Table XIV. 

On the average, more statistics were used in the expositions con- 
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Table XIV 

Ranking of Pamphlets by Proportion of Space Given to Figures 


Pamphlet 

Number Pages 

Pamphlet 

Number Pages 
with 
Figures 

Proportion 
Figures to 
Total 

Rank of 
Proportion 
of Figures 

Unemployment 

B 

29 

^5 

% 

86 

3 

C 

, . 26 

II 

42 

8 

D 

96 

58 

60 

5 

Peace 

E 

19 

16 

84 

4 

F 

59 

0 

00 

II. 5 

G 

20 

6 

30 

9 

Government 

H 

16 

0 

00 

II 5 

I . . 

■ . 30 

IS 

50 

6 

K . . , 

34 

15 

44 

7 

Money 

0 

• • 19 

18 

95 

I 

P 

. . 24 

5 

21 

10 

Q 

. . 29 

26 

89 

2 


cerning unemployment and unstable money than in the other two 
series of pamphlets. This was not always true: on the one side, in- 
flation was discussed without many figures in Pj and on the other 
side, when the cost of war was examined in E, graphs, figures, and 
tables were essential. 

No recognition was given in this count to the amount of figures on a 
single page. Practically, the ranking for quantity was more accurate 
than this apparent oversight would allow one to suppose. When fig- 
ures were presented on ninety-five per cent of the pages of a pamphlet, 
the discussion was necessarily statistical. Authors who introduced 
figures frequently were facile in their use of graphs, tables, and 
running comments. When figures occurred on less than fifty per 
cent of the pages, as was the case in half of the twelve pam- 
phlets, a sprinkling of figures interspersed through the context was 
the general rule. Among the pamphlets with less than fifty per cent 
of figures the number of figures on any one page in pamphlets ranking 
in popularity six through twelve was small 5 and the figures were 
in the context with two exceptions. Pamphlet K had one two-line 
table, and C had two simple graphs. 
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'Presentation of Controversial Issues 

It is assumed that in discussing almost any current issue there is 
more than one position to be taken. This assumption is made not only 
by theorists but also by the experimental readers. The readers often 
expressed themselves as “waiting for the catch/’ or as interested in 
trying to detect the author’s “frame of reference.” Their responses 
to the author’s position, stated or unstated, colored to some extent 
all of their other reactions. It is our belief that the problem of ad- 
justment between the bias of the reader and of the author is a major 
factor influencing judgments upon controversial literature. The word 
bias is used here in a psychological sense, to place emphasis upon the 
fact that every person has his inclinations and viewpoint. Each pam- 
phlet was examined for the author’s bias by three assistants and the 
writer, in order to determine the author’s method of expressing his 
viewpoint. One of the assistants (g)® was an intelligent man of 
volatile temperament, who disagreed with all comers and therefore 
could isolate prejudices of others. Another man (s) recognized the 
tricks of salesmanship after years of selling experience. The third 
(h) was working for an advanced degree in social science. 

These three assistants considered each pamphlet in turn, deciding 
which one of four descriptions of bias best characterized the author 
of that pamphlet. The types of bias exemplified when an author 
dealt with controversial issues were: 

1 . The author presented equal facts and arguments upon all sides of the issue. 
(Impartial) 

2. The author gave facts on all sides but indicated why he favored one side. 
(Fair partisan) 

3. The author presented his own view in the best light without indicating 
alternatives, (Salesman) 

4. The author painted only the defects of other positions and showed only 
the good points of his own views. (Prejudiced partisan) 

In the following table, XV, x represents the present writer’s judg- 
ment of the bias of each pamphlet, derived first from independent 
judgment and then from consideration of the opinions of the other 
three judges. The agreement of the raters upon' the types of bias of 
the pamphlets was close. Table XV shows the stand taken by each. 
In two-thirds of the pamphlets, raters either agreed upon the same 
type of bias or were one position removed. C was the only pamphlet 
on which there was a four-position disagreement. 

® The letters serve to identify the assistants in Table XV. 
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Table XV 


Judgment of Four Assistants Concerning Author’s Method of Presenting 
Controversial Subjects 


Pamphlet 

Author’s 
I. Impartial 

Position in Dealing with Controversy Was. 

2. Fair 3. Salesman 4. Prejudiced 

Unemployment 

B 

, . X h g S 



C 

.X h s 


g 

D 


X h g s ... 


Peace 

E 

h s 

X g ... 


F 


... s 

X h g 

G 


h s 

X g 

Government 

H 

g 

h X s 


I 

X h g s 



K 


... s 

X h g 

Money 

0 

X h g s 



P 

X h s g 


Q 

s g 

... X h 



* Tile positions are stated in full on p. 123. 
h, g, s «• three judges 
X final position, assigned by fourth judge. 


In assigning panaphlets to types of bias, it was necessary to con- 
sider the probable intent of the author as evidenced in his writing and 
to determine his general method of dealing with issues. There was 
some overlapping of types of bias in almost every pamphlet. An 
example of mingled types to which readers objected was contained 
in pamphlet I. There Socialism, Fascism, and Communism were dis- 
cussed with fair presentations of the strength and weakness of each 
economic system. The author of I was criticized by the readers for 
her partiality in the last section, which ended with the sentence, 
“Given such an opportunity, American workers would not be tempted 
by Communism, Fascism, or even Socialism. The New Capitalism 
would make America a better place in which to live.” In spite of this 
statement which fit the definition of a fair partisan, pamphlet I was 
assigned to an impartial position because of the author’s critical sur- 
vey of the three economic systems. 

In subsequent encounters with the quality of pamphlet here called 
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“type of bias,” the third position of “salesman” seemed to overlap 
with the second and fourth positions, and was omitted. Since it was 
impossible to reallocate the judgments of the assistants, the third 
position was included in this presentation. The revised method of 
judging three rather than four types of bias is defined in Chapter 
VI, page 196. 

Format 

Two assistants followed this outline in describing the format of 
each pamphlet; 

1. Cover 

a. Color of background and of type 

b. Decoration 

c. Size of book 

2. Quality of paper 

a. Texture 

b. Color 

3- Type 

a. Size and leading 

b. Opinion on legibility 

c. Changes in size for emphasis or headings 

4. Sections 

a. Number of editorial sections 

b. Average length and variation in length 

c. Interest of captions 

d. Method of treating quotations 

5. Number of pages in pamphlet 

The size of type was determined by a person who had had experi- 
ence in printing. She compared samples from each pamphlet with 
a complete style book of type size and leading. 

The description of format was utilized in several instances. Ex- 
amples of its use are given when two pamphlets are discussed in full 
later in this chapter, and again when standards are presented in the 
sixth chapter. 

If the experimental books remain available to the investigator, 
then an elaborate description of the kind indicated above is unneces- 
sary. Some description, however, is serviceable in later phases of the 
investigation, when the relationship between preference and a num- 
ber of factors is being sought. Information that cannot be obtained 
by looking at the books, and that might therefore be recorded in a 
description of format, is, size of type, area of book, number and 
length of sections. 
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Literary Quality 

As described in Chapter III (page 47) the attempt to rank the 
pamphlets, even as few as three at a time, on literary quality was 
unsuccessful. A list of possible components of literary quality was pre- 
pared by the assistants, but it seemed to be irrelevant to their judg- 
ments. The four assistants did not agree on their ranking of a single 
pamphlet. In two-thirds of the instances (eight pamphlets) the same 
book in a series of three was ranked i, 2, and 3 in literary quality by 
the judges, 

EXPERIMENTAL GROUP’s ANALYSIS OF PAMPHLETS 

The twelve experimental pamphlets were analyzed objectively by 
assistants in the manner just described. Some idea of their content, 
their literary quality, their appearance, and their methods of ap- 
proaching the reader’s interests was obtained. Scales or numerical 
descriptions were established for difficulty, quantity of figures, 
quantity of history, and presentation of controversial subjects. 

Reader criticisms supplied another method of learning about the 
experimental reading matter. In most instances, the forty-five readers 
were not asked to make the same type of objective analysis as the 
assistants. The methods of analysis that the experimental group did 
use will be briefly rehearsed to refresh the reader’s memory. 

Each member decided which of the three pamphlets he would like 
most to read and stated the reasons. This was termed snap judgment. 
He read the selected pamphlet and wrote a general criticism of it. 
Then he read it slowly, rating each section on a five-point scale, 
and giving his reason for the rating. After he had analyzed the three 
pamphlets in a series in this fashion, he ranked them in the order 
in which he liked them. He also ranked the three on six different 
qualities, the only analysis of the quality of the material required of 
readers in the entire experiment. 

The value of the general and section criticisms, both for indicating 
standards and for helping the investigator to form opinions about 
specific literature, has been presented before.* The utilization of the 
different ratings for interest were discussed when the Preference In- 
dex was given® and later when the reactions of sub-groups were 
analyzed,® The aspects of the readers’ analysis of pamphlets which 

*See Chap. IV, pp. no. 

'See Chap. IV, pp. 88. 

' See Chap. IV, pp. 99-102, 
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remain to be discussed are first, their snap judgments and second, 
their ratings of qualities of the pamphlets. 

Snaf Judgment 

We have already shown that there was little shift in the average 
ranking by the group of the four major topics with which the pam- 
phlets were concerned.'' What relation is there between prejudgments 
of famfhlets that have been briefly examined and opinions after 
careful reading of them? 

Out of a total of 228, twenty different reasons were given by the 
readers for prefeiTing to read given pamphlets first. The reasons 
were classified under the Combined List of Criteria. The table in 
Appendix I B provides a concise impression of the outstanding fea- 
tures of the pamphlets which were cited before reading them. 

The five principal reasons for rating a pamphlet as appearing most 
interesting were: 

1. Good title 

2. Familiar author or editor 

3. Interesting looking content 

4. Attractive cover or frontispiece 

$. Comprehensive, instructive appearance 

The snap judgment rank of a pamphlet within a series gave an in- 
dication of Its general attractiveness. The rank after reading showed 
whether the preliminary judgment was justified. Table XVI dis- 
closes the fact that, according to group averages, the pamphlets 
which appeared to be most interesting beforehand proved to be so 
after reading. Within the series on unemployment and on money, 
the ranks before and after reading were identical. The proportion 
changed somewhat for peace and for government. 

The most significant change between snap and final preference was 
that for pamphlet F. It had been well advertised and therefore ex- 
cited so much interest that twenty-three, or seventy per cent, of the 
experimental subjects who voted wished to read it first. It either was 
disappointing, or it was overbalanced by the other two pamphlets on 
peace, for only thirty-nine per cent ranked it most interesting after 
reading all three. 

The column in Table XVI showing the number of coincidences 
between individuals who read pamphlets first, and therefore pre- 
sumably preferred them in snap judgment, and ranked them first at 
the end, is surprising when the agreement between the snaji choice 

''See Chap. IV, pp. 102-04. 
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Table XVI 

Relation between Snap Judgment Preference before Reading and Final 
Preference after Reading Three Pamphlets 


Namber of First Choices Within Each Series of Three Preference 


Pamphlet 

Snap* 

Final t 

Coincidence ^ 

Rank 

Snap Final 

Unemployment 

B 

6 

6 

5 

3 

3 

C 

13 

11 

6 

2 

2 

D 

19 

21 

7 

I 

I 

Peace 

E 

5 

13 

- 

2.5 

1-5 

F 

23 

13 

- 

I 

i-S 

G 

5 

7 

4 

2.5 

3 

Government 

H 

9 

10 

5 

3 

3 

I 

19 

15 

5 

I 

2 

K 

14 

17 

5 

2 

I 

Money 

0 

8 

7 

- 

3 

3 

P 

16 

17 

- 

I 

I 

Q 

14 

14 


2 

2 


* Preferred before reading series, 
t Preferred after reading series. 

t Rated most interesting by same reader before and after reading (14% of all votes for 7 
pamphlets coincided). 

and later preference expressed by the group is recalled. Seven pam- 
phlets were examined for coincidence of rating 5 only fourteen per 
cent of the readers agreed on first choice before and after. If the 
findings at this point are valid, snap judgment provided a good clue 
to the way the grouf would finally vote, but was no indication of 
individual consistency. 

Reader Ranking on Six Descriftive Qualities 
As a last step in their comparison of pamphlets, readers were asked 
to use their best judgment in ranking a series of three for the fol- 
lowing six qualities : attractiveness in appearance, explanation of num- 
bers, quality of writing, simplicity, quality of explanations, and rela- 
tion to personal problems. The Interesting results are given in Table 
XVII. 
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Table XVII 

Mean Reader Ranking of Pamphlets on Six Qualities 


ni Unemployment 

Points 01 Compnnson 3 c D 

War and Peace Government 

E F G n I K 

Money 

0 P Q 

Attractive looking . . 2 

I 

3 

2 

1 

3 15153 

2 I 3 

Numbers explained . 2 


3 



3 3 151-5 

I 3 2 

Well written 2 

3 

I 

3 

I 

2 2 13 

3 I 2 

Easily understood . . 2 


3 


2 

3 123 

2 I 3 

Good explanations . . 3 
Touches personal 


I 

3 

i-S 

1-5213 

3 I 2 

problems 3 

I 

2 

3 

I 

2 151-53 

3 I 2 

Preference 3 

z 


3 

2 

I 312 

3 I 2 


I indicates best in each quality. 

3 indicates poorest m each quality. 

The averages upon which these ranks are based are in Appendix III E. 


The rankings were apparently made in good faith, for they cor- 
respond with known facts about the pamphlets. The general judg- 
ment of like or dislike for the pamphlet apparently did not blind 
readers to good or bad qualities: no pamphlets had the same mean 
ranking on all factors. On the other hand, some of the six qualities 
evidently were significant determinants of preference, since the reader 
ranks of the qualities often coincided with the rankings for preference. 
Three or more of the six ranks were the same as the preference rank 
for five of the pamphlets, E, I, O, P, and Q. 

How does a reader’s interest in content influence the other stand- 
ards that he employs in judging a pamphlet? We have been search- 
ing for an answer to this question in many directions. These data 
provide one sort of answer. The interest in the topic of unstable 
money was low; pamphlet P made a good use of many of the tech- 
niques for catching reader interest as rated by the readers, and was 
best liked in the series. The ranks for preference for the other two 
pamphlets on money likewise followed the majority of rankings on 
six qualities. But when interest in a topic was great, the favorite 
pamphlet seemed to have been determined, not by the superficial 
qualities on which readers ranked it but by the way in which it an- 
swered the questions and fulfilled the need of the readers. In the 
popular series on unemployment the favorite pamphlet, D, ranked 
low in three aspects and high in only two. Pamphlet G, in the 
series on peace, likewise ranked lowest in three of six aspects. 
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These were evidently liked in spite of their relatively unattractive 
appearance and their greater difficulty. 

A reasonable hypothesis is that on the average and for materials 
of doubtful interest value, effort should be concentrated upon utiliz- 
ing all of the devices for attracting and holding readers. But when 
readers are known to be deeply interested in the content of a book, 
less attention is necessary to details of presentation. Of course, deep 
interest is so rarely present that the search must be continued for 
methods of making ordinary reading matter appealing. 

Comparison of each of the preference ranks with the six qualities 
shows that only one of the six bases for reader rating closely paral- 
leled preference ranks. 


Table XVIII 

Comparison of Reader Ranks for Preference and for Quality of 
Explanation 


Pamphlet Preference Rank * Expl.ination Rank f 


Unemployment 

B 3 3 

C a a 

D I I 

Peace 

E 3 3 

F a 1.5 

I 1.5 

Government 

H 3 2 

I I I 

K a 3 

Money 

0 3 3 

P I 1 

Q 2 2 


* 1 IS most popuhtr. 1 1 h,TS best explanation. 


In Table XVIII three-fourths of the ranks of the pamphlets for 
preference and for quality of explanation coincided; none of the 
remaining fourth was reversed. Readers evidently believed that the 
manner in which explanations were made influenced their attitude 
toward pamphlets. 




EXPERIMENTAL READING MATERIALS 131 

Although each of the six qualities analyzed by the readers deals 
with an area investigated in the laboratory by judges, the only one 
which may be directly compared with the objectively determined in- 
formation is difficulty. The readers ranked each pamphlet in its 
series on whether it was easily understood. A comparable Difficulty 
Index was secured by the Winnetka formula. 

Table XIX 


Comparison of Two Different Methods of Ranking for Difhculty 


Pamphlet 

Ranked by Readers 
“Easily Understood” 

Ranked by Formula 
“Difficulty Index” 

Unemployment 

B 

2 

IT 

C 

I* 

3 

D 

3 

2 

Peace 

E 

1 

3 

F 

. . . . 2 

I 

G 

3 

2 

Government 

H 

. . . I 

3 

I 

. . . 2 

2 

IC . . . 

... . 3 

I 

Money 

0 

2 

3 

P 

I 

2 

Q 

3 

I 


* I is easiest. f 1 is hardest. 


Table XIX shows that readers spotted the pamphlets with the 
easiest vocabulary in three out of four series. In ^wo of the four 
series their opinion of the most difficult pamphlet coincided with the 
formula estimate. They ranked pamphlets differently from the 
formula in half of the cases. This difference in ranking is interesting 
because in one instance® opinions of difficulty expressed by readers 
seemed to correspond with their preferences better than the vocabu- 
lary and sentence computation did. Some of the factors named earlier 
in this chapter, such as complexity of concepts, use of abstract terms, 
or awkward presentation of data may have contributed to the differ- 
ent definition of difficulty upon which reader rankings were partially 
based. 

®See Chap. V, pp. 138!. 
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The procedure of asking readers to rank several comparable pam- 
phlets upon significant qudities was of particular value. The ratings, 
which were apparently made carefully, helped to validate some 
findings obtained in the laboratory examinations and to provide a 
basis for questioning other findings. 

While future curriculum research workers should not abandon 
objective laboratory investigations of the nature of materials, readers 
should be asked to collaborate in helping to determine their nature. 
Ranks by readers are preferable to reader comments in this phase 
of the experiment because the readers must establish a scale as they 
rank the material. This scale then helps to locate the degree of a 
quality that is present in a selected book. For example, readers ranked 
the pamphlets in the tmemployment series in this order of quality 
of explanations: i-D, 2-C, and 3-B. The degree of explanation in 
each of those pamphlets may tentatively be called good explanation 
in D, average in C, and poor in B. Although these degrees need to 
be checked by other techniques, they may m turn be used to verify 
other methods of estimating the quality of explanation, 

STANDARDS USED BY THE GROUP IN JUDGING EXPERIMENTAL 
PAMPHLETS 

Judges, who were assistants chosen from the experimental group, 
examined the twelve pamphlets in the light of eight standards. It was 
possible to assign some sort of numerical value to four of the quali- 
ties possessed by the pamphlets. Twelve degrees of difficulty, eleven 
degrees of quantity of figures, and twelve degrees of quantity of 
history were discovered in the twelve pamphlets. Four methods of 
presenting controversial issues were differentiated. 

The forty-five members of the group passed judgment on the 
pamphlets when they criticized them, and again when they in- 
dicated rank orders of preference for them. The readers’ comments 
will now be brought to focus upon the known quality of the mate- 
rial in order to see at what points the requirements of their stand- 
ards are indicated. 

The laboratory analysis of pamphlets just described established 
quantitative scales for four criteria, running from largest to smallest 
amount of the particular quality within the pamphlets. For instance, 
the pamphlets were ranked on difficulty from i to 12, hardest to 
easiest. 

Did the group appear to like particular degrees of those qualities 
better than other degrees? Was there any relationship between 
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preference and increasing difficulty, quantity of history, quantity of 
figures, or bias? The answer to these questions, stated briefly, is 
that (a) there was a tendency to dislike the easiest pamphlets and 
to approve the pamphlets that were median in difficulty; (b) there 
was a correlation between preference and lessening amount of his- 
tory; (c) there was a correlation between preference and lessening 
amount of figures; and (d) readers appeared to prefer pamphlets 
in which authors definitely stated and maintained a single position. 
These statements will be discussed in more detail. 

In order to give a quick resume of the findings of the preceding 
sections and to form a basis for further comparisons, the various 
laboratory analyses of pamphlets are brought together below in 
Table XX. 


Table XX 


Comparison of Preference and Four Qualities of Pamphlets 


Pamphlet 

Difficulty 

Index 

Quantity 
of History 

Quantity 

ofFigutes 

Type 
of Bias 

Preference 

Unemployment 

B 

3 

10 

3 

I 

9 

C 

II 

II 

8 

I 

3-5 

D 

5 

8 

5 

2 

1-5 

Peace 

E 

10 

2 

4 

2 

II 

F 

I 

12 

II. 5 

4 

S 

G 

6 

6 

9 

4 

1-5 

Government 

H 

12 

9 

" 5 

3 

8 

I 

7 

7 

6 

I 

6 

K 

4 

5 

7 

4 

7 

Money 

0 

9 

3 

I 

I 

12 

P 

8 

4 

10 

2 

3 5 

Q 

1 

I 

2 

3 

10 


Relation between qualities 

Relation between qualities and preference 

rho rho 

History and difficulty .. +.03 Preference and difficulty . -f.09 
History and figures . +.44 Preference and history ... — 46 

Difficulty and figures ... + . 16 Preference and figures ... — . 59 
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The correlations between pairs of qualities in Table XX show the 
extent to which they were interdependent in the pamphlets. No 
relationship was discovered between difficulty and quantity of his- 
tory. There was little relation between quantity of figures and vocabu- 
lary difficulty 5 the rank order correlation was .16. The quantity of 
history in pamphlets tended to increase in direct proportion with 
quantity of figures 5 rho equalled .44. The rest of the table will be 
discussed in the next few sub-sections. 

Difficulty 

The apparently insignificant correlation of .09 between difficulty 
and preference does not tell the whole story. When pamphlets were 
studied in series it was found that three of the four easiest were 
least liked, that the hardest ranked a little below average, and that 
every one of the median pamphlets in difficulty was the most popu- 
lar in its series. The trend toward preferring median difficulty when 
pamphlets in their series are studied is corroborated by referring back 
to the two Indexes in Table XX. Four of the six most popular pam- 
phlets rank 5, 6, 7, and 8 in difficulty. 

Historical Allusions 

The correlation between quantity of history and ranking of pam- 
phlets for preference was —.46. A more than random tendency was 
exhibited for a decrease in quantity of history to accompany increas- 
ing preference. 

Since the correlation between quantity of figures and quantity of 
history was .44, the two factors had somewhat the same ranking in 
pamphlets. Therefore it is possible that conclusions concerning one 
quality might apply to the other, or to the two elements working in 
conjunction. Nevertheless, there are enough differences in ranking 
to warrant some independent hypotheses. On the whole, those pam- 
phlets were preferred which had less history. Excellent use of his- 
torical information sometimes tempered adverse criticisms. The 
historical data were presented debghtfuUy in pamphlet P, in which 
the old custom of bartering was amusingly described, the growth of 
the use of bank money was sketched with a dramatic choice of words, 
and questions of interest to readers were asked which could only be 
answered by surveying the history of past depressions. Pamphlet 
P, ranking 4 in quantity of history, ranked 3.5 in preference. D, with 
more counted lines of history than any pamphlet except Q, was 
one of the two most popular among the twelve. Therefore, although 
the evidence favors material with little historical data, if the history 
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given is pertinent and well presented, this group of readers does 
not object to its inclusion. 

Presentation of Figures 

An attempt was made to isolate the factor of quantity of figures 
to a greater extent than was demonstrated with quantity of historical 
allusions. A large number® of sections with the constant element of 
presence of figures was compared with a large number having no 
statistical information. 

In the twelve pamphlets there were 202 sections. Of those, 85 
contained figures and 117 had none. The mean average preference 
for the sections with figures was 2.68 j the mean for those without 
figures was .24 less, 2.44. Since the standard deviation from the mean 
was only .21,^® this difference between the two means was a sub- 
stantial one, indicating that the sections without figures were distinctly 
more popular than were those containing statistics. 

The correlation of —.59 between preference and quantity of fig- 
ures and the difference of .24 in favor of sections without figures 
suggest that, constant proportions of other factors being present, 
material is better liked which has fewer figures. 

A qualification to that generalization is necessary, for the manner 
in which figures are presented has a great influence upon reader atti- 
tudes toward them. Sixty per cent of the pages in pamphlet D had 
figures in them; yet D was one of the two most popular pamphlets 
in the set. The other pamphlet ranking 1.5 in the Preference Index 
had figures in thirty per cent of its pages. Two pamphlets containing 
no figures ranked 5 and 8 in popularity. In the analysis of sections 
at the end of this chapter, one of the most popular and one of the 
least liked are found to deal with economic terms and figures. These 
do not, however, appear to be chance differences. On page 158 at- 
tention is called to a few of the differences in presentation which 
make one form appear uninteresting and another appear vital to the 
reader. More detailed studies of differences in the presentation of 
statistical material would be valuable. 


” The size of the sampling would tend to cancel other irrelevant influences. 

“ The readers ranked each section on a five-point scale of intei-est. The theoretical 
aveiage for each section might therefore range from i.o (superior) to 5.0 (dull). 
The actual aveiages showed a range of a little moie than one position; a.op to 3.15. 
The standard deviation of this range was .21. The median was 2 62. The average 
preference of sections without figures was 2.44, and of sections with figures was 
2.68. One-fifth of the entire range of the distribution lay between the two averages, 
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Presentation of Controversial Issues 

Each pamphlet was classified according to that one of four meth- 
ods which was used by its author to communicate his viewpoint to 
readers. Although numbers were assigned to each type of bias, these 
numbers were descriptive and could not be correlated with the 
preference ranks. Nevertheless, several comparisons between bias 
and preference may be made. 

The average preference for pamphlets assigned to each of the four 
types is indicated in Table XXI, 

Table XXI 

Relation of Author’s Method of Presenting Issues to Pamphlet 
Preference of Readers 


Type of Bias 

Pamphlet 

Preference 

Rank 

Mean Preference 
for Pamphlets in 
F.ach Type 

I. Impartial 



7.6 


B 

• 9 



C 

• •• 3 -i 



I 

6 



0 

. 12 


2. Fair partisan 



5 3 


D .. .. 

I s 



E 

II , 



P 

••• 3 5 


3. Salesman 



9.0 


H 

. 8 



Q ..• . .. 

. . 10 


4. Prejudiced 



4.5 

partisan 

F 

... s 



G 

1-5 



K .. . . 

... 7 



The number of pamphlets in each position is so small that the 
averages have no statistical significance. It is interesting, however, 
that the six pamphlets which presented viewpoints on both sides 
but expressed definite opinions of their own (types 2 and 4) were 
more popular than the six which either impartially dealt with all 
sides or did not admit the possibility of more than one side to the 
argument (types i and 3). 

The way to discover the criteria used by any specific group has 
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been illustrated by the preceding comparison of pamphlet prefer- 
ence with four qualities. Several difficulties were encountered in 
the course of the analysis. There were too few examples of treat- 
ment of each element to reach thoroughly reliable conclusions. The 
number of variables was too great to permit the definite isolation of 
any one, except in the instance where two hundred sections with and 
without figures were contrasted. The interrelationships among dif- 
ferent factors were sometimes difficult to disentangle. Within the 
limits of the data, however, enough has been learned to give clues 
to providing material for this group, clues which should be checked 
for corroboration at every opportunity. 

On the basis of the evidence just cited, if I were selecting curricu- 
lum materials for the Civil Works Administration group, I would 
tentatively provide them with the following: 

(a) Material that was preferably of a range of difficulty between 
school grades six and eight. That of moderate difficulty, without tech- 
nical language, would probably be preferred by the majority of this 
group. 

(b) Material that was not overloaded with figures. The necessary 
figures should be presented lucidly and cleverly. 

(c) Material that had few historical references, or historical ref- 
ferences with an evident significance. 

(d) Material that presented a viewpoint vigorously, recognizing 
alternative viewpoints but maintaining a single one, with or without 
slandering opponents. 

These four rules relate only to the standards analyzed here. Simi- 
lar rules for the use of the other thirty-four standards in the list are 
necessary before the best current-problem literature could be selected 
for the experimental group. 

STANDARDS CONTRASTED FOR TWO GROUPS DIFFERING IN EDUCATION 

The four principles enumerated above would guide the selection of 
curriculum materials to be supplied to the whole Civil Works Ad- 
ministration group. By following those principles the requirements 
of many members would be satisfied. But the recollection of the 
range in preference among individuals and small groups, described 
in Chapter IV, would cast doubt upon the possibility of meeting 
the needs of every member by the selection of a single book. 

There were some few pamphlets upon which most of the readers 
agreed j most of them liked pamphlets D, G, I, and P. One way of 
solving the problem of selection would be to give all of the mem- 
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bers the pamphlets that were liked by the majority. Another method 
would be to conduct an analysis of reasons for individual disagree- 
ment and an attempt to vary the selections with the tastes of differ- 
ent people. If the second alternative were accepted, the attack on 
the problem would be, first, to find the individuals that differed widely 
from the majority, and second, to discover why their reading prefer- 
ences varied. 

Six groups that might be supposed to vary from one another were 
studied.^ There was not much difference in opinion between men 
and women, or between people who had studied social sciences and 
those who had not. A wide difference in preference was detected be- 
tween eight people who had postgraduate degrees and eight who had 
been to college for two years or less. If any variation of reading selec- 
tions were permitted among the six groups studied, those groups 
which differed in college education should be given two kinds of 
material. The problem arising then would be the establishment of 
different standards for selecting the two sets of material. 

Contrast on Basis of Analysis of Pamphlets 

The squared difference between the undergraduate and post- 
graduates’ average ranking of pamphlets in series of three was 3.5.^^ 
Several factors in the background of the sub-groups have been sug- 
gested that help to account for the disagreement, factors such as 
differing educational and vocational experience, or differing personali- 
ties. What was there in the pamphlets which might produce such a 
difference in preference? 

In Table XXII the four analyzed qualities of the pamphlets are 
placed parallel to the average preference ranks of the two sub-groups 
that difer in education. 

An examination of the first choices of the less educated group re- 
veals some interesting facts. In three of four instances, the b^est liked 
pamphlet has the easiest vocabulary and the lowest quantity of 
figures and history. The fourth instance shows pamphlet E to be the 
easiest in its series but to possess more history and figures than the 
others. Evidently the members with less education appreciate the non- 
technical introductions to current problems. 

When the measure of difficulty determined by the readers is used, 
the postgraduate group ranks the most difficult pamphlet first in 
preference in every series. If the Difficulty Index is the instrument of 

“ See Chap. IV, pp. 95-10*. 

“See Table IX, pp. 100-01. 
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Table XXII 

Relation between Four Qualities of Pamphlets in Series of Three and the 
Reading Preferences of Two Groups That Differ in Education 


Prefer- Difficulty Piefer- 

ences Quan- Quan- -j-yng enee of 

of Two- a b. tityof tityof Post- Pam- 

Pamphlets year Diffi- Reader His- Fig- * grad- phlets 

College culty Evalu- tory ures ‘ uate 
Group Index ation Group 


Unemployment 









B 

2 

I 

2 

2 

I 

I 

3 

B 

C 

I 

3 

I 

3 

3 

I 

I 5 

C 

D 

3 

a 

3 

I 

2 

2 

1-5 

D 

Peace 









E 

I 

3 

I 

I 

I 

2 

3 

E 

F 

2 

I 

2 

3 

3 

4 

2 

F 

G .. 

Government 

3 

2 

3 

2 

2 

4 

I 

G 

H 

I 

3 

1 

3 

3 

3 

3 

PI 

I . . , . 

2 

2 

2 

2 

I 

I 

2 

I 

K 

Money 

3 


3 

I 

2 

4 


K 

0 

3 

3 

2 

2 

I 

I 

3 

0 

P 

I 

2 

1 

3 

3 

2 

2 

P 

Q 

2 

I 

3 

2 

3 


Q 


* See p. laj foi interpretation of numbers under types of bias. 

I indicates (a) most difficult, (b) most easily understood, most history, most figures, 
most popular 

3 indicates least amount of qualities 


measurement, the relation between difficulty and preference appears 
to be random. The postgraduate group usually likes those pamphlets 
with most or average amounts of history and figures best. 

Another contrast between the two groups lies in their attitude to- 
ward the author’s presentation of issues. Where there is a choice the 
postgraduates prefer pamphlets which are ranked as prejudiced or 
as “salesman” partisans. In no case is their last choice a prejudiced 
statement. The undergraduates, on the other hand, almost always 
prefer impartial statements, or ones in which both sides are presented 
by fair partisans. Two of the four pamphlets which are least liked 
by this group are prejudiced. 
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Contrast on Basis of Criticisms Made by Readers 

The approach to group differences so far has been made through 
the use of averages and generali2ations. The heart of the problem is 
contained in the reactions of individuals to specific materials. An ex- 
ample will be presented, to show how the different responses of mem- 
bers of the two sub-groups to the same stimiilus may be used to 
determine the different standards that the two groups erected. 

The point of widest disagreement in pamphlet preference detected 
anywhere in the study occurred in the rankings for preference of 
pamphlet H by the two sub-groups that differed in education. Most 
of the members of the two-year college group ranked H, “Roosevelt 
and the Constitution,” first in preference; the average ranking of H 
by this group of eight was 1.38. The eight members of the post- 
graduate group ranked it either two or three, with an average of 
2.43. This difference of 1.05^® between the average ranks of the 
two sub-groups was far higher than any other difference discovered. 
Why? 

All of the criticisms made by the two sub-groups in thie section re- 
actions were classified. Frequency of mentioning the fourteen chief 
criteria were ranked separately and were correlated for the better 
and the less educated groups. Notwithstanding their difference in 
judgment, the two groups used the same categories of reasons with 
the high rank order correlation of .95. An unusual opportunity was 
presented to watch the conflict between the same classes of reasons, 
expressed positively and negatively by two opposed groups. 

The frequency with which each type of reason was mentioned is 
shown in Graph 5. The group that liked the pamphlet made many 
more comments than the group that did not like it. This tendency to 
refuse to waste time in writing about something that does not seem 
worthwhile was noted once before when the so-called “careless” mem- 
bers of the experimental group were studied. 

Most of the reasons for judging the pamphlet were stated in the 
five standards which ranked highest. The post-graduates made 77 per 
cent of all their comments and the two-year college group gave 81 
per cent on the same five topics of quality of information, literary 
techniques, literary quality, techniques of controversy, and difficulty. 

The graph shows that the greatest difference of opinion occurred 
concerning the quality of information presented. The less educated 
group mentioned the excellent quality of information in H often and 
made comparatively few disparaging remarks. The more educated 

“ See Table IX, pp. loo-oi. 
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Graph 5. Frequency of Classified Positive and Negative Criticisms, Showing Why 
Two Groups Disagreed about the Inteiest of Pamphlet H. 


group made more negative than positive criticisms of the information. 
Members of the two-year college group, who were enthusiastic about 
H, said of its information: 

Discussion is of utmost importance today. I never knew this before. Out- 
standing revelation. Interesting information. 

Their unfavorable comments were: 

Doesn’t answer question in entirety. Conglomeration of names. No facts. 

They either found the information valuable or wanted more of it. 

The postgraduate group presented a smaller number of opinions 
on information, disclosing an opposite viewpoint. Two frequent ad- 
jectives were “superfluous” and “obvious.” They also criticized some 
of the information as: 

Not essential. Not very informative. 

Their commendatory criticisms were: 

An unusual thought. Roosevelt’s stand is interesting. Gives exact informa- 
tion. Pertinent additional information. 

Another contrast between the two groups occurred in their un- 
favorable comments upon difficulty. Each of them mentioned the 
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positive aspect of “interesting in its simplicity”} but most of the com- 
ments made by both groups on difficulty were negative. The post- 
graduates said. 

Too big a subject to be so briefly argued. Quite elementary. Author should 

use man-size language. Apparently foi kids. 

The undergraduates critiazed H thus: 

Not very elucidating. Not concrete. Too much generalizing. Indefinite. 

Very vague conceining Constitution. 

While the first group found H too easy, the second group needed 
more explanation and facts. 

The same contrasts could be drawn between comments of the two 
groups which were classified under each of the criteria. The two- 
year college group liked H because it contained interesting and im- 
portant information, because it made clear simple explanations, be- 
cause it was well written, and because the author’s reasoning was 
sound and his point of view agreeable to the readers. The post- 
graduate group reacted to the same pamphlet, under the same cate- 
gories, by stating that information was trite and non-essential, that 
the literary style was juvenile and boring, that the explanations 
were good only as far as the too short distance that they went, and 
that the author’s logic was ridiculous at some points. 

If I were differentiating between the curriculum materials that I 
planned to present to different types of the Civil Works Admin- 
istration members, I should take into account the apparent fact that 
the members with more schooling demanded more advanced treat- 
ment of topics. I would not attempt to give them a book written with 
a high school audience in view, whereas the less sophisticated non- 
graduates might enjoy such books. I should seek to gather more evi- 
dence concerning the nature of the variations among the groups at 
different educational levels. 

ILLUSTRATIONS OF PAMPHLET ANALYSIS 

It is the writer’s opinion that all of the information about an in- 
vestigated piece of literature, if scattered among various tabulations 
and interpretations, should be assembled into a condensed report. 
This report should present a description of the material, the results 
of any objective tests, and an organized analysis of opinions about 
the material. This type of report bears an analogy to the sociological 
case study, in which pertinent information is assembled in order to 
allow experts to judge the proper disposition of the case. 



EXPERIMENTAL READING MATERIALS 143 

The case analysis of two pamphlets in the present study will be 
given in this section. One purpose of this statement is an illustration 
of the method of reporting upon investigated literature. A second 
purpose is further exploration of standards of judging reading mate- 
rial. In order to discover whether the methods of analysis evolved 
in this study would throw into relief the reasons for the popularity 
of one pamphlet and the unpopularity of the other, two pamphlets at 
opposite ends of the Preference Index were selected. As a matter of 
fact, while those reasons are distinct at some points, at others they 
overlap. The similarity of the pamphlets in some of their qualities 
indicates the complexity of the problem of predicting popular or un- 
popular material. 

The Most Popular Pamphlet 

The most popular pamphlet is considered to be D, even though 
it received the same rank of 1,5 in the Preference Index as G. It was 
in the preferred series on unemployment, and was a surprising 
favorite in view of its length and its mass of facts. The popularity 
of D was apparently due to the importance of the discussion to the 
readers, themselves unemployed, and to its dramatic presentation. 

Content, Pamphlet D, “Unemployment and Its Remedies” by 
Harry W. Laidler. 

Section 

1. “The Jobless Millions.” Description of typical unemployed. Figures on 

number of unemployed since 1920. Ill effects of unemployment. 

2. “Some Types of Unemployment.” Seasonal, cyclical, technological types. 

Caused by shifts in industry, old age. Quotations and statistics. 

3. “Get the Facts.” Bureau of Labor Statistics finds facts. 

4. “Public Employment Exchanges.” Abuses by private agencies. Plea for 

government to take over all employment information. 

5. “Public Works.” Present lack of long-range planning. Need for public 

works “reserve” to be tapped in hard times. 

6. “Unemployment Insurance Abroad.” Full analysis of British system, with 

tables. Briefer explanation of German, Russian, other systems. 

7. “Unemployment Insurance for America.” Statement of various plans 

proposed for America: state vs. federal, contribution by industry, volun- 
tary systems. 

8. “Other Immediate Remedies.” Reduction of hours of labor, industrial 

stabilization, systematic tariffs, relief. 

9. “The Profit System and Unemployment.” Need for social planning, for 

socializing Industry. 
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10. “Summary.” Resume, with numbered points, of the proposals for 
icmedymg unemployment. “A Model State Unemployment Insurance 
Bill.” 

Snaf judgment frejerence. Fifty per cent of the experimental 
readers chose D to be read first of the three pamphlets dealing with 
unemployment j fifty-five per cent liked it best of the three after 
reading it. The reasons given for the snap judgment preference of D 
were: 


Fiequency 

Good title 6 

Looks comprehensive . . 5 

Interesting topic . . 3 

Attractive cover . . 2 

Related to reader interest a 

Looks instructive ... 2 


Frequency 

Clarifies problem 2 

Related to current prob- 
lem I 

Interesting style . ... i 

Size of pamphlet . ... i 

Appears impartial ... i 

Good chapter division . . i 


The title was next to the most popular of the twelve pamphlets. 
D appeared to be more comprehensive than any other. 

Laboratory analysis of six qualities of fomfhlet: 

Techniques for relating subject to reader. The 
point kept foremost in D was the need for relieving suffering. The 
author assembled a background of factual information} but he picked 
out the most dramatic facts, even in discussing the technicalities of 
the British unemployment insurance system. Two mechanical means 
of catching attention were direct quotations m dialogue, reported from 
court hearings, and italics in the midst of factual presentations which 
emphasized surprising or important findings. The arguments that 
pointed out the weaknesses of the present individualistic system and 
that showed in detail how socialization might be provided would ap- 
peal to a majority of this group of readers, themselves unemployed. 

Difficulty. The pamphlet ranked fifth among twelve and sec- 
ond among three on the Difficulty Index. It was placed just above 
eighth grade according to the Winnetka formula. The readers judged 
D to be the most difficult in its series. Their ranking of the pamphlet 
as harder than the formula ranking may have been due to the small 
type, the bad arrangement of pages, and the number of statistical 
data, rather than to their Impression of the difficulty of the vocabu- 
lary. 

Historical allusions. Although D had no allusions to 
history antedating the World War, it had a proportionately large 
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number of post-war references and was ranked eighth in quantity of 
history. There were ten diflEerent pages with historical data. Only 
three of the references furnished background for the discussion of a 
problem. The others were used for illustrative purposes. In the first 
section there were illustrations of unemployment in boom times and 
in the more distant past} the sixth section showed how unemployment 
pools in Europe operated. The background history was used to build 
an understanding of the American government’s methods of dealing 
with unemployment up to 1930, of the early provisions of the English 
unemployment act, and of poor relief in 1930. 

Presentation of figures. Although D had more pages 
with figures than any other pamphlet, the proportion compared to its 
total number of pages was only sixty per cent. D ranked fifth in this 
respect. Table XXIII shows the distribution of pages of figures in 
pamphlet D. 


Table XXIII 

Frequency of Types of Figures in Each Section of D 


Section 

Number 


Number of Pages Containing Each of the Following Four 
Types ofFiguies 


Running Dramatic Table 

Account Application 


Graph 



Total 


50 ■ 4 


Table XXIII showed that almost every section had a number of 
pages containing figures. Figures were most often incorporated in 
running accounts or discussions. The four instances of dramatic appli- 
cation occurred in the section presenting ‘‘some types of unemploy- 
ment.” There technological unemployment was illustrated by imagin- 
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ing a visit to a factoiy when a number of men were needed and again 
when few were needed, and by a comparison of the productive ca- 
pacity of a man ten years ago and today. Three of the four tables 
had two columns that needed to be compared in only one direction; 
the fourth table was a more complicated full-page chart showing the 
financial organization of unemployment insurance systems in various 
countries abroad. 

The most popular section in D contained five pages with figures ; 
the least popular had no figures. When the fiive more popular sections 
were compared with the five less popular, there were 35 pages with 
figures in the former and 34 in the latter. The presence or absence of 
figures had no distinguishable effect upon section ratings. 

Presentation of controversial issues. The three as- 
sistants who judged the author’s method of presenting controversial 
issues agreed in assigning D to type of bias 2, “The facts of all sides 
are presented, but the author shows why he favors one position.” A 
few readers thought that the pamphlet was unduly biassed toward 
Socialism (it was printed by the Socialist Party); but most of the 
comments of readers that related to the author’s way of dealing with 
controversial issues were favorable, considering him either impartial 
or a fair partisan in his presentation. 

Format: 

Cover of pamphlet — Blue, with black print. A cartoon of an armored figure 
backed by factories, crushing small scurrying men. 

Size — ^A little smaller than average: X 8 inches. 104 pages. 

Type — Chiefly 9 point and 6 point. Much type change for quotations, causing 
an inartistic page. Footnotes on almost every page. 

Divisions — ^Ten chapters, with about fifty sectional divisions. No blank half- 
pages between chapters. 

Reader ranks of six qualities of famfhlets. The experimental 
readers’ rankings of the six qualities were lower than might be pre- 
dicted from D’s popularity. It ranked best on “well written” and 
“good explanations.” The rank on “related to personal problems” was 
average. It was considered to be the least attractive in appearance, the 
most complex in its presentation of numbers, and the hardest. The 
readers seemed to like the pamphlet in spite of its unattractive fea- 
tures. 

Interest ranking. Pamphlet D ranked first in preference in the se- 
ries on unemployment, and 1.5 among the twelve pamphlets. The sec- 
tion ratings may best be seen in Graph 6. Half of the sections fell in 
the upper sixteen per cent of the total distribution of ratings, and 



EXPERIMENTAL READING MATERIALS 147 

only one dropped into the lower sixteen per cent. The introduction 
and the conclusion were considered not only the most interesting sec- 
tions in the pamphlet, but the two best liked among over two hun- 
dred. 



Graph 6. Average Rating of the Inteiest of Each Section in the Most and 
Least Popular Pamphlets 

Possible explanations for the drop in popularity of section 3 are 
that, in the first place, it was two pages long in comparison with an 
average of ten pages for the others. Second, it had only three or 
four lines of “human interest.” Third, its content did not appear to 
be as important to the layman as the rest of the content in the 
pamphlet. (See Content, p. 143.) 

Classification of section reactions. Graph 7 shows visually the 
readers’ opinions of the most and the least popular pamphlets. The 
most significant difference between the two columns is the drift to- 
ward favorable opinions for D and the almost even division of favor- 
able and unfavorable opinions for O. 
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When the half of the graph pertaining to pamphlet D is examined 
more closely, two positive standards by which D was judged are 
prominent. There was an overwhelming approval of the selection 
and quality of information 5 and the literary techniques, both dramatic 
and expository, were praised. More negative criticisms were leveled 
at the information than at any other standard. The graph depicting 
section reactions to D will have more meaning if typical criticisms of 
readers, drawn from some of the most frequent favorable and un- 
favorable comments, are cited. 



Graph 7. Frequency of Classified Positive and Negative Criticisms of the Most 
and Least Popular Pamphlets. (Based on information in Appendix III C ) 


The most frequent favorable criticisms of sections, classified under 
the following standards, were: 

Content: Infoimation (including timeliness, selection, and quality) 

Extremely useful information. New facts and figures. Explanation of in- 
teresting and helpful remedies. Very interesting review of plans. Problems 
thoroughly discussed. Fact and color well intermixed. Well stocked with 
worth-while information. 

Form: Literary techniques (including expository and dramatic) 

Rich with varied and interesting quotations, illustrations, and documents. 
Exceptionally good comparison. Several interesting examples of unemploy- 
ment. Scientific analysis. Clear explanation of existing conditions. 

Presenting controversial issues: Techniques of controversy (including argument, 
criticism, influence on reader) 

All alternative possibilities well covered. Comprehensive and construc- 
tive criticism. Splendid suggestions. I, too, favor unemployment insurance. 

, Interesting because point of view differs from my own. Citizen should 
know this to vote intelligently. 
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The most frequent unfavorable criticisms of sections, classified 
under the following standards, were: 

Content: Information (see above) 

Information old and inadequate. Incomplete information. Too many 
quotations. Gross misstatements. Some of the information is debatable. 
Form' Di'fficulty (including concepts, abstractness, condensation) 

Incomprehensible and muddled facts. Too long. The author states too 
little in too many words. Too much generalization and not enough concrete 
information at this point. Too many details presented. 

Content. Presentation of figures (including tables, graphs, figures) 

Too statistical. Tendency to be dry; too factual. Don’t care for tech- 
nical information containing facts and figures. First chart not clear, and rest 
of section too factual. Figures are confusing. 

General criticisms. More revealing than any of the piecemeal 
analyses are the general criticisms which the readers made of the 
pamphlets. Often those readers who were not successful in giv- 
ing full section reactions were voluble in their final remarks. A typical 
opinion will be given, and then the salient features of the positive 
and negative comments will be extracted. 

A typical opinion of pamphlet D 

I think that the author has presented the material in a logical and inclu- 
sive manner. He achieved what he desired to do, namely, made clear the 
meaning of unemployment, showed what has been and is being done to cope 
with the problem in other countries, and gave an intelligent treatment of 
possible American solutions. His material is authentic and represents an 
extended knowledge of the subject. As is so often not true, he presents the 
material in an Interesting manner. I gained much that was helpful in my 
thinking from reading the pamphlet. 

Content 

Fine mixture of fact and trimming, well documented. Full account of 
facts, though effort is required to read so many details. Material is authori- 
tative and represents an extended knowledge of subject. 

Though interesting, subject matter is too detailed and complex. Too ex- 
tensive quotations. Article tiresome only in spots that deal with statistics. 

Very colorful use of dramatic quotations. Pamphlet made interesting by 
picture of actual conditions. Rich in thought, well organized. Simple but 
forceful terms used. Excellent in analysis; gets to the root of the matter and 
threshes it out in smooth, understandable, and readable style. 

If study were more detailed, would be more interesting. Picture painted 
has statistical coldness; more vital picturizations of what figures imply should 
be given. Material presented logically, but not simply. 
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Author's display of mewpoint 

Case presented in as unbiassed and impartial a viewpoint as possible. Spon- 
sored by a political organization, it frankly announces its bias, however, facts 

are presented fairly, with no attempt to coerce the reader into the author’s 

In spots seems written in a socialistic ballyhoo fashion. 

The commendation of pamphlet D was by no means unanimous. 
It achieved its first rank position by pleasing many readers, rather 
than by making strong followers of some and enemies of others. It 
was well liked because the author wrote clearly and well, knowing 
how to set up a human bond between himself and his readers on 
every page. He gave them information that they wanted upon a 
topic that was interesting. Since he did not make an emotional appeal 
himself, he aroused few emotions in readers. 

The pamphlet had several weaknesses, which detracted from its 
interest for the majority of readers. Its type and page arrangement 
were poor. The large number of data produced a lack of interest in 
the less ambitious readers. It should be said in defense of the pamphlet 
on the latter point that it was written to serve as a source book rather 
than to interest the casual reader. That it succeeded in creating in- 
terest among so many readers is noteworthy. 

The Least Popular Pam-phlet 

Although O was the least popular pamphlet with the experimental 
readers, it had many good qualities. No pamphlet was included in 
this study which was not worthy of serious attention. This pamphlet 
was given a favorable mention in Herring’s “Twelve Inch Book- 
shelf.” [i, p. 8] It found staunch supporters among the readers in 
the experimental group. On the other hand, the lack of interest in 
the subject felt by these readers was serious j the author had not con- 
sidered this handicap sufEciently to surmount it for the experimental 
group. 

Content. Pamphlet O, “Inflation: What Is the Gold Standard.?” 
by Gordon Hayes. 

Section 

1. “Introduction.” Money is one of the most important institutions. Its 

history is old; its system is changing. 

2. “The Gold Standard.” Definition is elaborated with facts. 

3. “Bank Deposits as Money.” The comparatively large amount of money 

in the banks is shown by a graph. A table shows growth in loans to banks 

over a period of seventeen years. 
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4. “Gold Reserves.” The government holds most of the gold. When people 

lost confidence and withdrew money, the President issued an edict for- 
bidding withdrawal. 

5. “Bimetallism.” Double standard money has had a varied career in the 

United States since 1792. 

6. “Why We Use Gold and Silver.” Metal is superior to other less stand- 

ardized forms of wealth. 

7. “Gold Today.” The value of gold is a carry-over from earlier times. 

It is largely a reserve now. 

8. “The Merit of the Gold Standard.” The gold standard restricts the 

amount of money in circulation. It stabilizes foreign exchange. 

9. “Fiat Money.” Definitions. Examples of its use in England, America, 

and other countries arc given. 

10. “Inflation.” Definition. Inflation occurred in America at the time of the 
Great War. 

n. “Deflation.” Definition. Deflation occurred in America after 1929. 
Was tragic, 

12. “Why Prices Fell,” There was a decline in buying power and in loans. 

This lesulted in hoarding. 

13. “The Rise in Prices from March to September, 1933.” A price rise was 

caused by increased confidence in business and a hope of inflation. 

14. “The Difficulty of this Subject.” This discussion has indicated the in- 

tricacies of money. A citizen should be skeptical of any proposals. 

Snaf judgment freference. Pamphlet O was selected by twenty- 
one per cent of the readers, who thought that it looked the most 
interesting in its series; sixteen per cent rated it first after reading 
the series. The reasons given for the snap judgment preference of 
O were: 



Frequency 

Frequency 

Interesting topic . . . . 

• 4 

Attractive cover .... i 

Author a Ph.D. . . , 

a 

Simple I 

Looks informational . 

1 

Promising table of con- 

Interested in topic . . 

. . I 

tent I 


Laboratory analysis of six qualities of famfhlet: 

Techniques for relating subject to reader. The 
author made an educational approach to the subject, in a concise and 
matter-of-fact tone. Simple language with illustrative facts were used 
to touch the readers’ interest. The author showed the strength and 
weakness of every argument. His attitude was calm and scholarly, 
never fiery. 

Difficulty. O was easy; it ranked ninth among twelve pam- 
phlets and third in Its series according to the Winnetka formula. Its 
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grade placement was 6.7. When the readers judged its difEculty, 
they ranked O second in its series of three. Since the difEculty of the 
second pamphlet as determined by the formula was 6.8, an accurate 
discrimination between the difficulty of the two could not be expected. 

Historical allusions. The comparatively large number of 
historical allusions in O was indicated by its ranking third in quan- 
tity of history. The allusions were concentrated in three sections and 
in a fourth paragraph. One section illustrated the use of fiat money 
during Civil War times; the other examples all occurred after 1914. 
Section 4 gave the effect of the depression on banks. Section 5 sketched 
the history of bimetallism. Sections 9 and 10 included history as il- 
lustrations, first of fiat money, and second of inflation in America. 

Presentation of figures. O ranked first in quantity of 
figures; all but two of its pages had some sort of quantitative data. 
There were two graphs, one table, one example of dramatic applica- 
tion (illustrating the creation of a money market by borrowing from 
the bank), and sixteen cases of figures incorporated in the context. 

Presentation of controversial issues. The three as- 
sistants who judged types of bias all thought that O was impartial 
in the presentation of viewpoints. The fundamental facts were stated 
objectively; the points upon which different authorities take issue 
were not mentioned. 

Format: 

Cover of pamphlet — ^White, with black print. Centered cut showing a man in 
possession of tools of science and power looking over a tall city. 

Size — 4)4 X 8)4 inches. 19 pages of context, with bibliography and ques- 
tions in addition. 

Type — 12 point. Sections headed by heavier, larger type. 

Divisions — Fourteen sections, a little over a page each in length. 

Reader ranks of six qualities of 'pamphlet. The readers’ ratings 
of pamphlet O corresponded with the pamphlet’s lack of popu- 
larity. There were three third ranks: “well written,” “good explana- 
tions,” and “related to personal problems.” O was ranked first in 
its explanation of figures, second on appearance, and second on ease of 
understanding. 

Interest ranking. O was ranked twelfth in interest in the whole 
set, and third in comparison with the other two pamphlets on un- 
stable money. There is no questioning the fact that the readers con- 
sidered it dull if Graph 6 , page 147, is examined. There half of the 
sections were shown to be among the dull sixteen per cent; the other 
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half were not far removed from the division line. The two sections 
that had an average of 3.15 were the least popular sections among all 
of the pamphlets. 

In searching for reasons for the difference in interest of the sec- 
tionSj it was found that the five more popular sections had only eight 
pages of figures, as against twelve pages in the less popular. Half 
of the history was in sections on one side of the “dull” line, half on 
the other. Section ii, where the first appearance of a “human in- 
terest” phrase was noted under content above, had the lowest average 
of all. (See analysis of section, page 143.) Section 12 was the most 
interesting in the pamphlet, with its account of “why prices fell.” 
Section 13 was almost as well liked j it described the reason for a 
subsequent rise in prices. The interest in sections 12 and 13 may be 
explained by the relatively new ideas presented there. 

Classifcation of section reactions. Graph 7, which appears on page 
148, shows that the experimental readers made a few more favorable 
than unfavorable section comments upon pamphlet O. The section 
criticisms of pamphlet O were better distributed among all of the 
standards than were those for D. Only three criticisms were con- 
spicuously frequent: the commendation of good explanations and 
examples under literary techniques, and the unfavorable criticisms 
of difficulty and of the quality of information. Two standards drew 
no comments: emotional appeal and the use of the author’s name. 
(The latter was mentioned twice under snap judgments, in referring 
to the author’s Ph. D.) The four most often mentioned standards 
will be given, two of which had many comments of both a favorable 
and unfavorable nature. 

The most frequent favorable criticisms of sections, classified under 
the following standards, were: 

Form: Literary techniques (including dramatic and expository) 

Good illustrations. Interesting description. Short but clear explanation. 
Striking fact mentioned. 

Form; Dipculty (including terms, concepts, concreteness, and condensation) 

All technical terms omitted, but equivalent meanings clear. Technical subject 
broken up and made readable. Keeps one interested by simplicity of style. 
Clearly understandable. 

Content: Information (including timeliness, selection and quality) 

Interesting because of the present talk about inflation. The information given 
here helps me to understand what the government is faced with. Shows the 
operation of the system of money clearly. Author does well to point out 
lack of knowledge within the field of monetary systems. 
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The most frequent unfavorable criticisms of sections, classified 
under the following standards, were: 

Form; Difficulty (see above) 

Not enough definition of terms; not clear. Uninteresting explanation of a 
term. Author is writing down; too elementary. Much too involved. Get- 
ting out of my depths again. Writer should have been constantly aware of 
the fact he is dealing with a technical subject about which the reader knows 
nothing. 

Form; Literary quality (including grammar, simplicity, vitality, organization, 
adult approach) 

Seems to be having all he can do to watch his step grammatically. Could use 
more simple, less involved sentences. Getting nowhere. Author not coherent. 
Development is slow. Points not made forcefully. Interesting topic, but 
author talks like a Sunday-school teacher. Must be written for a high school 
pupil. 

Content: Information (see above) 

Too many necessary background details omitted. Not enough information. 
No doubt very learned, but as far as I am concerned, not very informative. 
States only well-known, elementary facts. Emphasizes the very obvious. Old 
problem of bimetallism retold. 

General criticisms. The general criticisms contained many con- 
flicting opinions, as may be seen in the excerpts below. 

A typical opinion of pamphlet O 

In the author’s effort at extreme objectivity, he tends to sacrifice style to 
method of approach. This pamphlet is written in too general a style. At 
times it simply “slurs over” important issues. However, it is clearly written, 
readable, and easily understood. 

Introduction 

Too long in getting started. Too abrupt; author does nothing to stimulate 
the reader’s interest. 

Content 

Technical subject treated in simple manner. Contains major facts in re- 
gard to money. Historical development of money especially interesting. 
Diagram helps to illustrate story of inflation. 

It is brief and does not give facts that are essential for understanding. 
Gives the reader a half-cocked view of a difficult subject. Not enough ex- 
planation of charts and tables. Use of large figures without any significance. 
Form 

Presented clearly and concisely. Vital facts vigorously presented. Author 
writes in a smooth, simple style. 
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Lacks fire to make it appealing. Author reads like a thinking machine 

rather tlian a living person. Dry, heavy, and academic. 

Conclusion 

A sense of bewilderment is created by reference to intricacy of subject. 

No definite conclusions arc reached. 

It is true of pamphlet O, as it was of pamphlet D, that there was 
no unanimous agreement on it. More than half of the comments 
about O were favorable; but there were more unfavorable criticisms 
about O than about any other pamphlet. 

The chief faults of O were that it had no dramatic touches, that 
it was not related to the interest of the experimental readers, and 
that it was not in touch with their requirements for a discussion of 
inflation. Almost half of the negative criticisms of difilculty said 
that it was too easy, with too elementary a selection of information. 
The other half said that it was too hard, with too great a condensa- 
tion and a lack of adequate information. 

Some readers found its information valuable, with its comprehen- 
sive, definitive survey of a field. The statistical information was well 
presented. The explanations were clear and were simply stated. 

ANALYSIS OF MOST AND LEAST POPULAR SECTIONS 

In conclusion, let us examine one further set of data in order to 
intensify the contrast between the material that was popular and the 
material that was unpopular in the opinion of Civil Works Admin- 
istration readers. Two sections from pamphlets D and O, sections 
that happen to be similar in a few respects and widely different in 
others, will be given the reader to see the material that the experi- 
mental group criticized. 

The most popular and the least popular sections among all the 
twelve pamphlets will be quoted. They were selected after examin- 
ing all of the interest ratings. The method of rating each section on 
a five-point scale and of obtaining an interest average for every sec- 
tion was described previously. The two most popular sections among 
the entire two hundred had an average of 2.09. They were both 
in pamphlet D; one was the introduction. The average of the two 
least popular sections was 3.15. Both were in O; one was section ii. 
Those two will be presented. 

The well-liked section from D tried to awaken a consciousness of 
the meaning of unemployment, while the disliked section from O 
showed the effect of deflation upon the nation. Both authors wished 
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to arouse in the readers an impression of uneasiness concerning the 
condition of the country; but one did it by means of incidents and 
dramatized figures and the other by statistics. The author of O was 
cramped for space, having less than a page to discuss his topic, 
while the author of D wrote more than five pages. 

The first section must be condensed greatly, chiefly by omitting 
stories and a table. The second section will be repeated in complete 
form. 

The Most Popilar of Two Hundred Sections^ from Pamfhlet D 

Section i. The Jobless Millions (condensed) 

I am writing this pamphlet during the second year of the industrial de- 
pression. For the last two winters millions of men and women have been 
walking from factory to factory, from shop to shop, from office to office, 
pleading for the privilege to work but denied that privilege. “A thousand 
men are sleeping nightly on the lower level of Michigan Avenue,” reads 
one of the numerous repoits from Chicago. “The lack of employment has 
knocked the bottom out of things . . . family morale has been destroyed 
. . . families that used to get along simply disintegrate and take to drinking 
and begging,” writes Helen Hall. 

Stories sucii as these arc multiplied by the thousands. Just how long the 
line of unemployed is it is impossible to state with any degree of accuracy. 
The census taken late m January, 1931, led to an estimate of 6,030,000 out 
of a job, able to work and looking for it. This number, however, failed to 
include those on fort time, a grouf whech frobably amounts to another stx 
or seven milkon. In the best years, when plants were working to capacity, 
there was a margin of unemployment amounting to more than a million men. 
(Table was given showing average mmlmums for unemployment from 1920 
to 1927). And these figures, the National Bureau of Economic Research 
insists, were rock-bottom minimum figures. 

The greatest tragedy of unemployment is its psychological effect upon the 
worker. A vice-president of a public utility was discharged as the result of 
a merger. He was compelled to go to a sanitarium to recover from a subse- 
quent nervous breakdown. Constant irregularity of employment is bound 
to leave its mark on the worker. Humiliation, want and hunger frequently 
leave their indelible effect on mothers and children. 

The author of this section possessed a rhythmic style, in which 
the repetition of patterns and ideas led the reader forward easily. 
He gave several illustrations, then proceeded to an italicized gen- 
eralization, When he presented figures he said in effect after each 
repetition, “Go back to look at them, for they are important.” He 
skillfully introduced an admirable figure of industry — a vice-presi- 
dent — as the dramatic link in an argument, and ended the section with 
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a plea for mothers and children. No reader could follow through 
illustrations to generalizations, back to concrete examples and then 
to inevitable outcomes, without being aware of the intensity of the 
unemployment problem. 

The writer of the section on deflation in pamphlet O had a dif- 
ferent purpose from the Socialist author of D. He was preparing 
an educational explanation of inflation, of the sort that an audience of 
high school students might study. The experimental readers would 
not accept his purpose, but wished to be shown the underlying sig- 
nificance of inflation and to learn how it might affect them. The 
least popular section follows. 

The Least Pofular of Tvoo Hundred Sections y front Pamfhlet O 

Section II. Deflation (in toto) 

The word deflation also has many diffcient meanings, but the general mean- 
ing is that prices are falling sharply. We were in such a period from the 
summer of 1929 to March, 1933. During that time, the wholesale price 
level fell from 150 on the basis of 100 for 1913 to 80 in the week ending 
Match 4., 1933. 

It is diflicult to exaggerate the tragic consequences of such a marked decline 
in prices in less than four years. The income of the fanners of the nation 
was cut m half as a result of this decline in prices, falling from eleven bil- 
lion dollars in 1929 to about five billion dollars in 1932. Producers of raw 
materials, other than agricultural products, found their profit margins wiped 
out and curtailed production. Manufacturers were faced with declining 
markets. Total industrial production fell from an index of 125 in the sum- 
mer of 1929 to less than 60 in the summer of 1932, a decline of more than 
50 per cent. It became more and more difficult to pay debts — bankruptcies 
increased, mortgages were foreclosed, and the number of unemployed rose 
to a staggering proportion of the total working population. No one knows 
how many persons were unemployed in the spring of 1933. Estimates run 
as high as 15,000,000 and 17,000,000 which is approximately a third of the 
working population of $0,000,000. [s, pp. 17 f.] 

The author has no particular literary style; his short sentences 
neither flow together nor build to a climax. Only two phrases in 
the section indicate any attitude toward the data, “tragic conse- 
quences” and “staggering proportions.” In most of the other sen- 
tences the bare facts must arouse their own images in readers’ minds. 
He expects figures standing alone to convey a vivid meaning to lay- 
men. For instance, the phrase at the end of the first paragraph, 
“the wholesale price level fell from 150 . . . to 80 . . .” would 
paint a picture of panic to one who understood the technical verbal 
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process of economics} but to one not accustomed to seeing any sig- 
nificance in figures the phrase would be meaningless. 

Given the same assignment faced by the author of O, the author 
of the preceding section from D might have defined the term “de- 
flation” by individualizing a few of the facts, letting separate stories 
build into one or two well-selected generalizations. He would have 
told the reader the meaning of large figures in terms that were part 
of the reader’s experience. 


SUMMARY 

The experimental reading materials were analyzed for two dif- 
ferent reasons. The particular problem of this study was furthered 
in that the information needed to complete an analysis of some of 
the standards was secured. In addition, some of the ways to study 
books and to synthesize the findings were demonstrated. 

The process of building standards described in this chapter started 
with the laboratory analysis of several qualities of the experimental 
pamphlets. For instance, the quality of difficulty was studied by 
applying a formula to see what proportion of it existed in each pam- 
phlet. Next, the ratings by readers of the same qualities were 
analyzed. As many of their ratings as possible were used: their 
rating of pamphlets for difficulty was compared with the formula 
ranking and found to differ from the formula on half of the pam- 
phlets. This type of analysis presented a graphic picture of some 
of the qualities in the pamphlets. 

The next problem was to discover what the readers thought of 
those qualities. This was solved by comparing the qualities of the 
pamphlets with the preferences of readers for them. Using diffi- 
culty as the example again, the Difficulty Index was placed parallel 
to the Pamphlet Preference Index to see what level of difficulty 
was best liked. The comparison showed that ordinarily the pre- 
ferred pamphlets were those that were average in difficulty. This 
generalization described the standard of difficulty that was applied 
to the pamphlets by the readers. Similar generalizations were ob- 
tained for the standards of presentation of figures, historical allu- 
sions, techniques of controversy, and appearance. 

After the standards that were applied by the whole group were 
determined by examining the average of group preferences, a refine- 
ment was made. The diflEering requirements of sub-groups were 
investigated. Two groups whose formal educational experience dif- 
fered hy at least three years were found to select the most interest- 
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ing pamphlets on quite different bases. The group with fewer years 
of schooling liked pamphlets that had few figures, little history, and 
a simple style, while the group with more years of schooling required 
pamphlets that were more advanced in each of those qualities. 

The last step in the procedure of studying standards is to assemble 
all of the available information about each, including the criticisms 
offered by readers. Chapter VI is devoted to this task. When the 
present chapter had cai-ried the analysis of standards as far as it could, 
it resumed the task of studying the experimental materials in order 
to provide a pattern for synthesizing and evaluating the information 
collected about them. 

The most and the least popular pamphlets were presented as “case 
studies” of the experimental literature. Enough information was 
assembled about each to show its composition, to indicate reasons for 
its popularity or lack of it, and to evaluate its qualities. Finally, 
the most interesting section and the least interesting were contrasted 
in order to diagnose the strength of one and to suggest remedies for 
the weakness of the other. 
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CHAPTER VI 


STANDARDS FOR SELECTING READING MATERIAL 

The value of knowing the standards by which readers judge books, 
and the ways of discovering and defining some of those standards 
have been presented from various angles in the first five chapters 
of this book. In Chapter II it was shown that, if reading is to have 
the place that its importance warrants in adult education, books which 
hold the interest and answer the questions of readers must be sup- 
plied. An intelligently directed effort must be made to find books 
that are adapted to specific groups. In Chapter III three necessary 
steps in the selection of reading material were indicated: discovering 
aspects of literature that make it popular or unpopular, analyzing 
selected material in the light of these aspects, and studying the re- 
quirements of specific types of readers concerning these aspects of 
the material. In the author’s opinion, the threefold process not 
only will lead immediately to selection of good study material, but 
ultimately may simplify the selection of all study materials by pro- 
viding serviceable rules to this end. These steps were illustrated by 
the example of one group and one set of pamphlets in Chapters IV 
and V. 

In order to complete the presentation of standards of book selec- 
tion, the information collected while performing the above three 
steps will be organized in terms of the standards which it explained. 
In addition to utilizing information from the present study all of the 
available research upon the various standards will be introduced. 
The knowledge obtained from those sources will be presented under 
each of the thirty-eight criteria that readers used in evaluating con- 
troversial public problems, 

STUDIES OF ADULT READING INTERESTS 

Most exploratory studies of adult reading interests have stopped 
at the point where this study begins. They are concerned with a pre- 
liminary problem: determining the subjects that adults enjoy reading 
about. Their findings are a necessary introduction to the present prob- 
lem, namely, discovering how adults evaluate the subjects that they 
i6o 
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read. Those preliminary studies show how to delimit the field of 
selecting a subject which a specific group will study. Some of the 
best known of them will be mentioned briefly. 

One type of study is concerned with the amount and the quality 
of reading which adults do. The type may be illustrated with 
analyses performed by Rasche, Monroe, and Witty. Rasche in- 
vestigated the kind of material that was most frequently read by a 
special group; Monroe and Witty were interested especially in the 
variations between reading habits of different groups. 

Rasche [8]* studied the reading habits of seven thousand voca- 
tional school pupils between fourteen and eighteen years of age. 
In the field of magazine reading, the men most frequently read 
magazines dealing with fiction, general topics, and science. The 
highest per cent of women read fiction, home, and general maga- 
zines. Students who had been enrolled in the school six months 
read less fiction and more general magazines than did those who had 
just entered the school. In newspapers, more than seventy per cent 
of the men read city news, comics, sports, and world news. Seventy 
per cent of the women read only the comics and stories. 

Monroe [7] interviewed two hundred seventy residents of Chi- 
cago. On an average, they spent ninety minutes each day in reading. 
The amount and type of reading varied with the occupation of the 
reader. 

Witty [12] asked mothers of preschool children about their read- 
ing. When he classified the results in terms of years of formal 
education, he found that there was little difference between the 
amount and kind of reading that mothers at various educational levels 
did, with one exception. Mothers in the least educated group read 
less than those in the better educated groups and spent their reading 
time upon a different type of literature. But college graduates read 
the same type of material as high school graduates. 

When Witty correlated reading habits with the family income, 
he found a positive correlation between increasing income and in- 
creasing time spent in reading. The mothers in homes with an in- 
come between $3000 and $7500 read significantly more than groups 
with lower or higher incomes. 

A second type of study deals with the general or specific subjects 
that are interesting to selected groups. Examples of this type are 
the studies made by Roberts and Davis, and by Waples and Tyler. 

* Throughout this chapter numbers in brackets refer to bibliographical references 
to Chapter VI, as given on pp. 199-202. 
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Roberts and Davis [9] gave a questionnaire to two hundred fifty 
teachers, in which the teachers were asked to recall the books they 
had read during the previous year. The investigators grouped the 
various returns under the heads of professional, literature, current 
publications, and biography. The authors analyzed fluctuations in 
mention of topics in relation to teaching experience, subjects taught, 
age, living conditions, academic training, hours spent reading weekly, 
and presence or absence of a professional library. 

The best example of the detailed examination of reading interests 
is Waples and Tyler’s study of “What People Want to Read 
About” [ii]. They asked ten different types of groups to check 
those topics which they would like to read about from a carefully 
prepared list of 117 topics picked from recent magazine articles. The 
expressed interests of all these groups were tabulated from many 
angles, and showed that the most important factors influencing choice 
of topics were, in order, sex, schooling, occupation, age, environment, 
and time spent per week in reading. All of the sampled groups were 
interested in the topics of international attitudes and problems and 
personal hygiene. There was no topic that was unpopular with 
every group. 

The third type of study of reading interests is concerned with a 
somewhat different phase of the analysis of reading interests. After 
discovering what subjects people are interested in reading about, 
the next problem is to find why they are interested in those particular 
subjects. Several different approaches to a solution of this problem 
are exemplified in the studies by Montgomery, Hart and Holbrook. 

Montgomery [6] attempted to find underlying psychological mo- 
tives for reading. His method was to ask about six hundred people 
why they did recreational reading. They said to satisfy curiosity, 
to relax, to get emotional satisfaction, to secure culture, to gain vicari- 
ous experience, to encounter vivid description, to get background, 
and to idle away time. 

Studies have also attempted to discover the reasons why certain 
writing is successful. For example, in an unpublished master’s thesis. 
Hart [4] selected ten novels which had been the most popular over 
a decade. Book reviewers’ opinions of them were studied. They 
were then carefully analyzed for action, characterization, structure, 
treatment of form, and style. Conclusions about some factors which 
were related to the novels’ popularity were drawn from these “case 
studies.” 

A delightful study of the reading qualities demanded by woods- 
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men was made by Holbrook [5], who has “logged all over the 
country” and knew his group. The study was a repoi't of experience 
rather than of research. Holbrook named the magazines that log- 
gers read in order of preference, gave case studies, if so technical a 
term could be applied to his narratives, and left no doubt about 
the kind of fiction to supply this kind of reader. He said, “Most 
loggers like their literary meat red. They want a story with black 
curly hair on its chest.” Again, “They strongly favor the gallon 
hat, two-gun cowboy school of books.” 

The studies which have been cited are typical of the best analyses 
of (a) the amount and quality of reading that specific groups do, (b) 
the subjects in which readers are interested, and (c) the general 
qualities that readers like to find in their books. The logical next 
step is an examination of experiments upon the special qualities to be 
found in books. The experiments that have been performed by 
other investigators, where such experiments have been located, will 
be summarized under the treatment of each quality in the following 
pages. Research studies have been made of seven of the thirty-eight 
standards in the Combined List of Criteria. These experiments are 
described under the following criteria. A 2, fresentation of figures} 
B I, difficulty; B 3 a, dramatic techniques; B 4 a, tyfe; B 4 c, 
illustrations; B 4 f, title, and B 4 g, make-uf of book. 

The fact that comparatively few standards have been subjected to 
experimentation indicates a lack of research upon fundamental prob- 
lems of selecting literature to interest and educate adults. If there 
are studies of standards which the writer has not indicated in succeed- 
ing pages, they are too obscure to be of much influence upon further 
research, for all of the ordinary and some extraordinary sources were 
searched for contributory studies.® This study has suggested prob- 
lems that might be investigated under several of the standards.® 

The standards discussed in the following pages are in many stages 
of formulation. Some of them are little more than bare topical head- 
ings with a few reader criticisms to amplify and define them. Others 
have been studied sufficiently to be more clearly delineated and to 
have some guides to their use erected. All of them need to be worked 
out much more fully for differing groups and for varied types of 
reading materials. 

* The identification preceding this and the following criteria refers to the position 
in the classification outline. 

“The investigation of sources was not carried on systematically aftei June, 1934. 

'See Chap. Ill, pp. 64-67. 
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DESCRIPTION OF STANDARDS 

Exflamtion of Headings Used in Describing Standards 

A uniform method of describing each standard for selecting read- 
ing material has been adopted. The importance quarter and the 
positive-negative ratio will be stated for each standard. Then the 
experimental evidence will be assembled, first from other studies and 
next from this investigation. Opinions of writers upon the impor- 
tance of the standard will be cited and interesting comments that the 
experimental readers made in regard to it will be stated. Finally, 
the divisions into which the criticisms of pamphlets were classified will 
be given. 

In many instances information was not obtained for some of the 
headings described below. Under such circumstances the heading will 
be omitted without comment. Each standard is introduced by a para- 
graph that gives its chief characteristics, either in terms of the kinds 
of criticism that are classified under it or in terms of significant facts 
about it. 

Importance quarter. The importance quarter was described fully 
in Chapter IV, page 83. The reader will recall that the thirty-eight 
standards were ranked in order from most often to least often men- 
tioned. The best indication of a standard’s proper position was 
thought to be, not the specific rank, but the quarter of the distribu- 
tion in which the rank fell. If a standard is in the first quarter, then 
it is one of the nine that were most often mentioned. Frequency 
of mention and importance are here considered to be synonymous} 
therefore a standard in the first quarter is among the first nine in 
importance, and a standard in the fourth quarter is among the lowest 
nine in importance. 

Positive-negative ratio. There is no way of telling from an exam- 
ination of the favorable and unfavorable reactions listed under a 
standard whether or not most of the reactions are favorable. For 
example, under A i, page 166, did the classified criticisms usually 
suggest that historical references were interesting and important, or 
were they perhaps more frequently considered dull and too abun- 
dant.? The method of discovery is to consult the positive-negative 
ratio. The ratio is based upon the section reactions and indicates the 
proportion of them that were favorable and the proportion that were 
unfavorable. Returning to historical allusions, the positive-negative 
ratio is 83+ : 17—. This ratio means that eighty-three per cent of 
the criticisms were listed as interesting, important, or easy, and that 
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seventeen per cent were regarded as dull, too lengthy, or wrongly 
interpreted. 

Evidence from other exferiments. Any experimental studies which 
bear upon each standard will be introduced under the appropriate 
headings. In some areas where many investigations have been made, 
only a few typical ones will be cited. 

Evidence from this study. Any data that may have been contrib- 
uted by this study will be reported. Generally the data will be 
assembled from discussions in Chapters IV and V and from the 
Appendix, but occasionally data will be presented for the first time. 

The evidence from this study must always be interpreted in the 
light of the facts about the experimental group.^ Thus, a full re- 
port incorporated in historical allusions would read: Forty-five people 
with a college education, who had been unemployed for some time 
but who were largely trained for professional jobs, two-thirds of 
whom were men, two-thirds of whom were under twenty-seven, 
seven-eighths of whom were unmarried, one-half of whom attended 
no church, and four-fifths of whom were interested in reading about 
most of the subjects treated in the experimental pamphlets, tended 
to dislike historical allusions. The information about the group will 
not be repeated in the citations of evidence from this experiment, but 
should be recalled every time that the group’s preferences are men- 
tioned. Since previous experience is a primary determinant of pref- 
erences, the experimental group’s background is important. Groups 
with other backgrounds will have other preferences both in subject 
matter and in standards. 

Etdfert opnions. Writers, adult educators, publishers and libra- 
rians all have opinions about the standards that operate when readers 
select books. Some of their opinions were collected in this study and 
are classified here under the appropriate standards. The sources 
are identified in Appendix I A, where the expert opinion list is given 
in full. The number in the footnote following each opinion refers 
to the position in the outline in Appendix I A. 

Interesting reader comments. Pertinent criticisms by readers, 
selected from either general or section reactions, will be presented 
whenever they aid in clarifying the definition of the standard. 

Classification of reader criticisms. The readers gave almost forty- 
five hundred criticisms of the experimental pamphlets, which were 
organized under thirty-eight different divisions of standards. Many 
decisions were made concerning the assignment of criticisms to the 

* The facts are presented in Chap. IV, pp. 84-87. 
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various categories. Gradually a definitive outline was prepared to 
show all of the shades of expression that could be included under 
each standard. All of the criticisms given by the experimental readers 
can be classified under one or another of the categories listed in the 
“classification” beneath the standards. The classification is stated 
in this text for two purposes; first, to define the standards more pre- 
cisely, and second, to aid others who wish to use the same standards 
in their classification of a large number of readers’ criticisms. 

Favorable criticisms are given in a column on the left of the 
page, and unfavorable criticisms are given on the right. Different 
shades of the meaning of the standard are assigned to different lines. 
All of the standards described here are listed in Chapter IV, Table 
V; the letter and number to the left of each standard refer to the 
classification of the standard in that outline. In addition, the outline 
of the four major divisions of the list will be given at the beginning 
of each division to aid orientation to the position of the standards. 

Stmdards for Selecting Reading Material 

A. CONTENT 

Under content are classified all of the readers’ criticisms that are 
primarily concerned with subject matter. Since we are dealing with 
qualities of content rather than with the context itself, we ask, not 
what subject matter should be presented to a given group but what 
the qualifications should be of any subject matter that is presented. 

A. Content 

1. Historical allusions 

2. Presentation of figures 

3. Treatment of subject matter 

a. Timeliness 

b. Selection of information 

c. Quality of information 

A I. HISTORICAL ALLUSIONS 

Although the experimental readers made comparatively few criti- 
cisms of history, as the importance quarter showed, the history that 
was present in the pamphlets apparently was a deterrent to their 
preference for pamphlets. 

Every one of the readers’ criticisms that includes history in its word- 
ing or that obviously refers to a passage containing abundant his- 
torical allusions should be classified under historical allusions. 
Importance quarter: 4 (least important) 

Positive-negative ratio: 83-}- : 17 — 
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(No evidence from other e^rperiments) 

Evidence from this study: 

In performing the pamphlet analysis of the number of historical 
allusions, history was defined as dating from the beginning of the 
present presidential term (1932).® There was a definite though un- 
reliable tendency for the experimental readers to prefer those pam- 
phlets which contain fewer lines of history. The rank order corre- 
lation between preference and number of historical allusions was 
—.46.® Two of the pamphlets with most history employed the 
allusions in an interesting manner and were among the most popular 
pamphlets. Historical data may not be a handicap when they are 
interestingly presented in pleasing content.'^ 

There was practically no relationship between difficulty and quan- 
tity of history. Rho in that case was .03. Quantity of history and 
quantity of figures tended to increase together, as shown by a corre- 
lation of .44.® 

(No expert opinions) 

(No interesting reader comments) 

Classification of reader criticisms under historical allusions: 

(1) Interesting treatment (i) Dull 

(2) Important (2) Too much given 

(3) Easy (3) Wrongly interpreted 

A 2. PRESENTATION OF FIGURES 

The presence of figures in exposition is known to have a definite 
effect upon readers’ attitudes toward the material. Although students 
often demanded figures in answer to questions, they always wanted 
the figures presented with clarity and with an indication of their 
significance. 

All reader criticisms concerning figures and those few about facts 
that refer definitely to quantitative data are included under the 
presentation of figures. Remarks about facts in a broader sense are 
classified under exposition (B 3 b). 

Importance quarter: i 
Positive-negative ratio; 54-|- : 46 — 


“ See Chap. V, p. 119. 

“ See Chap V, Table XX. 
’ See Chap. V, p. 134. 
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Evidence from other exferiments: 

Magill [ax] performed the valuable service of tabulating the fre- 
quency of occurrence of various graphic forms in current reading 
matter of the “non-specialist” type. He sampled general maga- 
zines, newspapers, and automotive instruction sheets until he had 
what he believed was a representative sampling. The rankings for 
the frequency of the five major groups of graphs was (approxi- 
mately) . 

Maps — 10 Architectural drawings — .6 

Mechanical drawings — 2 Electric circuit diagrams — .05 

Graphs — I 

Washburne [23] tried out diflrerent forms of the same material 
on over two hundred children and, in light of his findings, derived 
a list of rules for presenting figures. Typical rules are: 

When m doubt about the difficulty of data, never use a paragraph or a picto- 

graph. 

For complex comparisons with time constant, use a bar giaph. 

For extremely simple comparisons, use a pictograph. 

For specific amounts, use round numbers (500, not 532 or five hundred), 

A controversy was started in the Journal of the American Statistical 
Association by W. C. Eells [20] over the merits of circles rather 
than bars to show proportions in graphs. In spite of general opinion 
to the contrary, he found students to be more accurate, faster, and 
more interested when reading and interpreting circles than when 
reading bar graphs. His findings were substantiated by Croxton and 
Strykker [19], with larger groups of judges and more carefully 
paired diagrams. In both cases, however, the diagrams were un- 
labeled and were presented separately. This fact was pointed out 
by R. von Huhn [22], who questioned the validity of the findings. 

Evidence from this study: 

The rank order correlation between preference for pamphlets and 
number of pages with figures was — .59.® There were several in- 
stances in which rank order of figures and of preference were widely 
different. Both pamphlets D and O were high in quantity of num- 
bers, but one was most popular and the other was least. 

Comparison of all pamphlet sections containing figures with those 
having no figures revealed an important difference between preference 
averages in favor of sections without numerical data.® 

'See Chap. V, Table XX. ’See Chap. V, p. 135. 
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Comparison of the most popular section in all of the pamphlets 
with the least popular section showed that, if figures were made in- 
teresting by pointing out their significance or by dramatizing them, 
they were more successful than if the reader had to interpret them 
himself/® 

Exfert ofimons: 

People want facts rather than theories when they are looking for 
help with problems/’- Avoid overweighting written material with 
complicated graphs and details/^ Statistics should be supplemented 
with simple diagrams/® 

Interesting reader comments: 

Charts and figures do more explaining than the reading matter does. Large 

figures which have no significance for the reader are used. Reader interest 

is killed by use of graphs and charts so near the beginning. 

Classification of reader criticisms under presentation of figures : 

(1) Easy (i) Too technical; heavy 

(2) Full explanation; (2) Too much tabulation; not 

complete enough explanation 

(3) Interesting; good (3) Boring 

A 3. TREATMENT OF SUBJECT MATTER 

Treatment of subject matter refers to all reader criticisms of the 
general content of the material under the following three sub- 
headings. 

A 3 a. TIMELINESS 

All of the experimental material was timely in the sense that it 
was written within three years of the time of the experiment and 
that it discussed cuiTent issues. The readers’ opinions about its time- 
liness was in some cases a different matter, as the negative ratio sug- 
gested. 

Importance quarter. 2 
Positive-negative ratio: 36-I- : 64 — 

Exfert ofinions: 

Reading should be based on and connected with the readers’ imme- 
diate interests.’® Magazines should note and deal with “topics in- 
sistently clamoring for attention.”’® 

’“See Chap. V, pp. 156-5S. “See Appendix I A, B 3 a. 

” See Appendix I A, B 3 c, See Appendix I A, A 3. 
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Classification of reader criticisms under timeliness: 

( I ) Important current topic 

(2) Information is old 


A 3 b. SELECTION OF INFORMATION 

Selection pertains to the criticisms o£ the inclusion or exclusion of 
certain information by the author. It differs from the following 
standard, for there opinions about the quality of information which 
has already been selected will be segregated. 

Importance quarter: I 
Positive-negative ratio: 41 -|- : S 9 — 

Ewdence from this study: 

“Interesting content” (i.e., well selected) ranked second among 
all of the reasons for preferring to read certain pamphlets first (snap 
judgment) The two pamphlets ranking highest with this comment 
were F and H. Some of the comments on the information expected 
in H and F were: 

Interest in subject 

Expect to find adequate information 

Every one should know all about this subject 

Suggest something new 

Interesting reader comments: 

Explanation of facts not generally known by average reader. Con- 
ditions reviewed too familiar to reader to hold his attention. 

Classification of reader criticisms under selection of informa- 
tion: 

(1) Well selected 

(2) Information or angle new 

(3) Pertinent 

(4) Good subject 

(5) Important subject 

A 3 C. QUALITY OF INFORMATION 

The criticisms classified under quality of information are first, 
those that describe the content without qualification, and second, 
those that are definitely concerned with the kind of information in 
pamphlets. Criticisms that contain information as one of their minor 
elements are generally classified elsewhere. For instance, the com- 

“ See Appendix 1 B. 


(1) Superfluous 

(2) Information trite, ordinary, 
obvious 

(3) Irrelevant 

(4) Dull subject 

(5) Trivial 
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ment, “Information just stated, not explained” is placed under ex~ 
fository techniques rather than here. 

Importance quarter: i 
Positive-negative ratio: 46-]- : 54 — 

Evidence from this study: 

In the snap judgment preference, the reason “comprehensive” 
ranked sixth in importance.^® The pamphlet with the most com- 
ments on this point was D. Those given were: 

Appears to have lots of facts 
Seems to have new information 
Appears thorough 

Appears to deal with problem as a whole 

Seems to cover field both intensively and extensively 


The quality of information ranked second in importance in the 
general criticisms of pamphlets by readers.^® 

Classification of reader criticisms tmder qualitv of information: 


Statement of subject treated wj 
count of causes of deflation.” 

(1) True; scholarly 

(2) Thorough 

(3) Instructive; helpful 

(4) Comprehensive; authentic 

(5) Excellent analysis; gets at 
root of difficulty; points 
specific 


qualification, i.e., “An ac- 

(1) Inaccurate; wrong 

(2) Inadequate; thin in ideas; 
superficial; too detailed 

(3) Author handles subject in- 
capably 

(4) Causes not analyzed; author 
falls down on details 


B, FORM 

The standards classified under form are: 


B. Form 

1. Difficulty 

a. Terms 

b. Concepts 

c. Abstractness 

d. Condensation 

2. Literary quality 

a. Organization 

b. Form and grammar 

c. Vitality 

d. Rhythm and fluency 

e. Concise, simple language 

f. Adult approach 


3. Literary techniques 

a. Dramatic 

b. Expository 

c. Figures of speech 

4. Format 

a. Type 

b. Size of book 

c. Cover and illustrations 

d. Footnotes, index, and 
table of contents 

e. Sections and captions 

f. Title of book 

g. Make-up of book 


‘ See Appendix I B. “ See Appendix I C. 
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B I. DIFFICULTY 

Although difSculty has received more experimental attention from 
other investigators than has any of the other technical phases of read- 
ing, the factors which actually cause difficulty in reading are still 
elusive. Important causes of difficulty are considered here to be diffi- 
culty of terms and of concepts, degrees of abstractness and of con- 
densation. 

Evidence from other experiments: 

Until recently attention has largely been centered upon the gram- 
matical elements of components of the printed page, particularly 
words. Counts of several million words have been made to determine 
which occur most often and are therefore probably the best under- 
stood by general readers [27]. 

Dale and Tyler [31] have studied other possible causes of diffi- 
culty by analyzing not only words but complexity of sentences, num- 
ber of personal pronouns (as a measure of informality), per cent of 
monosyllabic words and other elements which were found to show 
little statistical relationship to difficulty of comprehension. 

After much experimentation with the relative influence of different 
factors upon difficulty, Washburne and his assistants combined three 
of the most important mechanical factors into a formula and chart 
[29]. The formula is intended to be applied to children’s books. 
Gray has prepared a more complex formula, consisting of five ele- 
ments with two constants for predicting difficulty for adults with 
limited reading ability [33, pp. 137-139]. 

Two theses illustrate an attempt to analyse difficulty in broader 
terms. Carroll [30] derived his experimental data from children 
who read directions for performing acts and answered questions about 
the directions. He listed five main factors producing errors in com- 
prehension: 

“How-many-ness (arithmetical calculation) stimulates a random search for 

the answer.” 

Prepotent factor or word 

Sentence containing conditional clauses 

Too compact sentences or statements 

Ideas merely implied, not clearly stated 

Irion’s study [34] is significant in that it analyses the factors of 
literary comprehension carefuUy and prepares tests for each of the 
points discovered. His analysis follows: 
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Literary 

comprehension 


lAi. 


lA. Narrow 
reading 
comprehension 


Word 
knowledge 
Knowledge 
of expres- 


3. Ability to 

I. Reading get specific 

comprehension facts 

IB. Broad IBi. Ability to 

comprehension points and 

conclusions 

II. Ability to interpret, IIA. General in- 
to make applications formation 


He found that word knowledge was an important item in reading 
comprehension, but that it was not nearly as important in the total 
(or broad) comprehension as in the narrow (or specific) comprehen- 
sion. 

The most detailed work on difficulty of reading for adults has been 
done by Gray and Leary. Eighty different variants which might be 
related to difficulty were enumerated, of which sixty-four were ca- 
pable of quantitative analysis. The method of analysis was to count 
the number of occurrences of each variant in a reading passage and 
to correlate the frequency of the percentage with a difficulty score 
made on that passage by seventeen hundred adults. Five elements 
with the highest correlations were combined in a formula for pre- 
dicting the difficulty of books. The formula was immediately uti- 
lized to analyse the difficulty of 350 books that are commonly used 
in adult education [33, pp. 12-13, 339-350]. 


Evidence from this study: 

Difficulty was surveyed from several angles in this study. First, 
reader criticisms were classified under four heads, terms, concepts, 
abstractness, and condensation. Second, the twelve pamphlets were 
ranked according to the grade assigned by application of the Win- 
netka formula.^^ Third, the ranks for difficulty were correlated 
with ranks for pamphlet preference, with a correlation of .09.^® 
Fourth, an analysis of preference for the easiest, average and hardest 
pamphlets in each of the four series showed that the easiest were 
generally liked least, and the average in difficulty liked best.^® Fifth, 

" See Chap. V, Table XII. “See Chap. V, Table XX. 

“See Chap. V, p. 134. 
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readers ranked the pamphlets in each series in order of difficulty. 
They recognized three of those ranked easiest by the formula and 
assigned the same ranks as the formula to three other pamphlets. 
Their order of ranking disagreed with the formula on half of the 
twelve pamphlets, suggesting that these well educated readers had 
another basis for judging difficulty than the vocabulary and sentence 
structure on which the formula for school children was based.^® 

Sixth, a group of eight members that had been to college two years 
or less was compared with another sub-group possessing postgraduate 
degrees to discover the relation of differences in pamphlet preference 
to variations in pamphlet difficulty. The contost between the two 
sub-groups was marked when the readers’ own estimate of difficulty 
was the basis of comparison. The less educated group rated the 
easiest pamphlets most interesting in every series and the hardest 
pamphlets least interesting in three of the four series. In contrast, 
the group with postgraduate degrees preferred the hardest pamphlet 
in each of the four series. We may conclude that even when the 
education far exceeded that needed for comprehending the pam- 
phlets, groups with different educational backgrounds disagreed in 
their attitude toward easy and hard pamphlets.^^ 

B I a. TERMS 

The readers in this study made surprisingly few criticisms of the 
difficulty of terms, considering the relative importance of words as 
found in most of the studies of difficulty. All specific comments on 
definition of words are grouped under terms. Here are also placed 
appraisals of colloquialisms. The general comments upon difficulty 
are classified under the next standard. 

Importance quarter; 3 
Positive-negative ratio' zz-\- : 78 — 

Evidence from other exferiments: 

Three types of studies of terms, from the many that have been 
made, have a bearing upon adult reading. One is a discovery of the 
vocabulary range in ordinary reading material for adults. The second 
is an analysis of the difficulties that adults have in reading. A third 
searches for the influence of vocabulary upon difficulty. 

The first type is illustrated in a study by Witty and La Brant [28]. 
They calculated the percentage of words from the various divisions 
of the Thorndike Word List in each of twelve adult books, includ- 

“See Chap. V, p. 131. 

“"See Chap. V, pp. 138!. 
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ing inferior and superior fiction. Ninety per cent of all of the words 
were in the first five thousand of frequency. Zane Grey and Harold 
Bell Wright’s masterpieces possessed a vocabulary difficulty equal to 
books by James Barrie and Edith Wharton, and were more diffi- 
cult than other books rated as classics. 

Second, Kepner [24] shows a technique for obtaining estimates of 
vocabulary inadequacies in specific fields. Seventeen social studies 
teachers recorded all of the words which in the course of teaching 
it was evident were not understood by fifty per cent or more of their 
students. Kepner analyzed the lists that the teachers submitted. He 
found that forty-eight per cent of the words were less frequent than 
Thorndike’s first ten thousand; that fifty-three per cent were un- 
specialized words, while the remaining forty-seven per cent dealt 
with the language of the social studies. 

The third type is best exemplified by Gray and Leary [33, p. 
1 1 5], who found that out of forty-four potential elements of diffi- 
culty, four of the five most important elements dealt with easy or 
hard words. In regard to the four elements related to terms, the 
correlations between average test scores made by all readers and 
the percentage of easy words in selected passages was .522, between 
scores and number of words not known to ninety per cent of sixth 
grade pupils .520, between scores and number of easy words .513, 
and between scores and number of different hard words .496. 

Esufert ofimons: 

Any difficult words should be given repeated definition and illus- 
tration.^® Writing should on the whole be non-technical. But occa- 
sional technical words are liked, if they are made meaningful.®^ 

Classification of reader criticisms mder terms: 

(1) Colloquial language good (i) Objection to slang 

(2) Good definition of terms (2) Not enough definition 

(3) Easy words (3) Do not understand terms 

Bib. CONCEPTS 

The dictionary definition of a concept as an idea or a complex of 
characters implies a realm of - experience to which the reader is or is 
not accustomed. All general remarks pertaining to difficulty of read- 
ing materials are classified here. 

Importance quarter: r 
Positive-negative ratio: 17-}- : 83 — 

“ See Appendix I A, F 3. “ See Appendix I A, F 4. 
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Evidence from other experiments: 

Murphy [35] analyzed the richness of concepts in the vocabulary 
of twenty people, ten of whom were good readers and ten of whom 
were poor. He concluded: “The concept appears to play a role in 
reading ability from the point of view of organization, clarity and 
accuracy. The investigation has not demonstrated, however, that 
general variations or differences in the richness of concepts are of 
any particular importance as variants of reading abdity.” 

Expert opinions: 

Authors should take into account differences in concepts as distinct 
from vocabulary limitations.^* 

Interesting reader comments: 

Basic theme vividly and clearly presented in non-technical language. 
Classification of reader criticisms under concepts: 

(1) Subject well defined (i) Failed to define topic 

(2) Too hard to read (2) Too elementary 

(3) Non-technical (3) Technical; heavy 

(4.) Plainly written; easy style (4) Not clear 

B I C. ABSTRACTNESS 

The degree of abstractness of reading material is probably closely 
related to difficulty. In spite of the importance of abstractness, no 
studies that bore directly upon it were found. As classified here, 
the aspects of abstractness are largely negative. Its positive forms 
may be seen in some of the definitive phrases under concepts, vitality, 
dramatic techniques, and techniques of relating subject to reader. 
Importance quarter: 3 
Positive-negative ratio: S 5 -|- : 45 — 

Expert opinions: 

Take as concrete forms of the great problem as you can find.^° 
Interesting reader comments: 

Study made interesting by picture of actual facts and conditions. Newspaper 
inserts and examples help to visualize thought. Interesting transition from 
general problem to specific example. Too general to be interesting. 

Classif cation of reader criticisms tmder abstractness: 

(i) General statement followed (i) Too abstract; vague; too 
by concrete examples general 

“ See Appendix I A, A 4. “ See Appendix I A, B 4 b. 
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Bid. CONDENSATION 

Condensation may be defined as the degree to which ideas are 
crowded together. The relation between condensation and difficulty 
is influenced more by the previous experience of the reader than 
most of the other standards in this list. A reader who has never be- 
fore seen a given idea expressed may consider a paragraph where 
it is discussed too condensed j someone who has a passing acquaintance 
with the idea may say that the treatment is excellent 5 and a student 
of the subject might prefer a single sentence to the developed para- 
graph. 

The criticisms classified under condensation were frequently en- 
countered in the experiment, students who were familiar with the 
concepts in a pamphlet considered it too wordy or repetitious, while 
readers with no background for the reading wanted more explana- 
tions, or said that the ideas were too closely crowded together. 
Those criticisms may be seen in the classification. 

Importance quarter' i 
Positive-negative ratio; i8-j- : 82 — 

Expert ofinions: 

It is important to recognize that it is not sufficient just to state 
a problem. Each person must understand and feel the problem as it 
looks to others.^® Writing should be repetitious.^^ 

Interesting reader comments: 

Says nothing, and says it in a diffused way. A bushel of chaff with a hand- 
ful of ideas. Repetition makes the writing more effective. No cramming 
of materials. Explains object in a few woids. Too garrulous. Big subject 
crowded for space. Touches too many subjects. 

Classifcation of reader criticisms under condensation: 

(i) Explains point in few words (i) Too many words; too brief 

(2) Too much repetition 

(3) Too crowded in ideas; 
hurried 


B 2. LITERARY QUALITY 

The general classification, literary quality, might also be called 
the “style of writing,” since most of the criteria of style that are 
taught in a course in writing are included here. The first two topics 
under literary quality are related to the formal aspects of writing, the 

“ See Appendix I A, B 2 b. 

^ See Appendix I A, F 3. 
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second two to vividness and skill in expression, and the last two to 
special qualifications of the style of expository material. 

B 2 a. ORGANIZATION 

Under organization are classified reader comments relating to the 
arrangement and to the unity or coherence of a piece of writing. 
Importance quarter: 3 
Positive-negative ratio; 37-|- : 63 — 

Interesting reader comments: 

Author deviates and leaves reader at a loss. Manifest coherence and unifica- 
tion of main theme. Author deceives reader by keeping facts scattered. 
Author belatedly picks up thread of argument in obscure manner. Arrange- 
ment of material uninteresting. 

Classi-fication of reader criticisms under organization: 

(1) Shows unity (i) Disjointed; incoherent 

(2) Direct, to the point (2) Main topic sidetracked 

(3) Systematic arrangement; (3) Lacks organization; uninter- 

good correlation esting arrangement 

B 2 b. FORM AND GRAMMAR 

In. the minds of the experimental readers, grammar pertains to the 
use of words, and form to the structure of sentences or paragraphs. 
Good grammar was assumed and was not commented upon; but 
good form was occasionally mentioned. 

Importance quarter; 3 
Positive-negative ratio; 32-I- : 68 — 

’Evidence from this study: 

One of the twelve pamphlets was written in language that was 
considered powerful and effective by one group, but labeled tab- 
loid, slang, or bad grammar by another. A sample chosen at ran- 
dom from the pamphlet follows: 

However, the crash is a crash. Day after day, stocks are going down. The 
little fellows become panicky. Soon they sell for eighty cents what they paid 
one hundred cents for. Stocks keep toboganning. Pray, can I have sixty, 
fifty, forty, thirty, twenty, ten cents for the dollar I paid you, cries {sic) 
the poor little fellows with their aU in stocks or on stocks. [50, p. 24] 

Probably no other pamphlet brought such sharp reactions as K, 
from which the above sample is extracted. It is evident that if a 
writer wants to draw attention, he will talk colloquially, without 
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much consideration of sentence construction. Here are two readers’ 
opinions of pamphlet K: 

(1) A rip-snorting, two-fisted pamphlet that has something to say and says it 
1 don’t entirely agree with the solution, but the pamphlet is good, gooa!/ 
I have enjoyed it heartily. 

(2) Of all the pamphlets I have ever read this is the worst. Exaggeration 
is rampant, and too often appeals to the heart instead of tlie head. The 
reading is terribly boring. 

ClassifcaAon of reader criticisms under form and grammar: 
Simple statement concerning: presence of question, premises, example car- 
ried through several paragraphs, climax, etc. 

(1) Interesting form (l) Too much outline form 

(2) Sentence structure good (3) Sentences poor, too long 

(3) Poor grammar 

B 2 C. VITALITY 

Vitality is a term borrowed from the American Library Associa- 
tion to denote that “indescribable something” without which a book 
is boring and with which it is charming. Often it is impossible to 
preserve for microscopic examination the most important emotions 
that a book induces. Vitality has not been dissected, but is represented 
by descriptive phrases in the classification. 

Importance quarter; I 
Positive-negative ratio: 47-I- : 53 — 

Expert opinions; 

Finally there is this most essential quality — ^vitality. Mere correctness of lan- 
guage is not sufficient. The book that will hold the casual reader will reflect 
the enthusiasm of the author in his subject and will bear witness of his 
understanding of the psychology of his reader.*® 

Interesting reader comments: 

Racy style holds interest. Good style; reads like a travel book. Subject 
dry; no attempt to make it interesting. Style is inflexible; reading difficult 
because of monotonous presentation. 

Classification of reader criticisms under vitality: 

(1) Well worded statements; (i) Poor style; badly written 

good style 

(2) Spicy; colorful; racy; (2) Shoddy; formal; dead; 

conversational dry; stereotyped 


‘ See Appendix I A, C 4. 
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B 2 d. RHYTHM AND FLUENCY 

Rhythm and fluency were important according to the opinion of 
experts; but the readers mentioned the standard infrequently. 
Importance quarter: 4 

There were only four section comments, 2-]- and 2 — 

Exfert ofinions: 

When writing for immature people, rhythm is a most essential 
feature.^® 

Classification of reader criticisms under rhythm and fluency: 

(1) Progresses smoothly and evenly (i) No progression 

(2) Moves rapidly (2) Does not move rapidly 

B 2 e. CONCISE, SIMPLE LANGUAGE 

The criterion, concise, simple language, has been well defined by 
Walter Hines Page in the excerpt given m “expert opinions.” 
Reader opinions are classified under this head when the comment is 
concerned with simplicity in connection with enjoyability of reading 
rather than with difficulty. 

Importance quarter: 2 
Positive-negative ratio: 88-f- : 12 — 

Exfert Ofinions: 

And when it came to style — ^terseness, directness, simplicity. He loved one 
or two syllabled Anglo-Saxon words. Walter Page was the enemy of the 
diffuse. He would say, “Do you realize that the story of the creation of 
the world, the biggest thing that ever took place, was told in a single para- 
graph?”®" 

Eighty-six and two-fifths per cent of all the worldh fights and arguments 
occur because the contestants have failed to define the words over which they 
clash. The great tragedy of violence is that when the dead and wounded 
are counted it will often be discovered that many of the slain need not 
have died at all if only they had understood the other fellow’s language. . . . 
What this nation and every other needs is a good collection of five-cent 
words. Half-dollar and two-bit words make for misunderstanding and 
strife.®^ 

Classification of reader criticisms under concise simple language: 
(l) Clear; concise; easy to read (l) Style too hard; too involved; 

no force 

“ See Appendix I A, C i. 

“ See Appendix I A, C 2. 

“ See Appendix I A, C a. 
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B 2 £. ADULT APPROACH 

The importance of this standard is 3, rather than 4, because of 
the frequency with which expert opinion cited the need for an adult 
approach. 

The criticisms that are directed toward the author’s patronizing 
attitude are classified under adult approach. Another type of ob- 
jection to the author’s style, expressed as “too elementary,” is 
classified under difficulty of concepts (B i b) rather than here. 
Importance quarter: 3 
Only 2 reasons, both negative 

Ex-pert opimons: 

Talk not down but across to the reader. Treat him as an intelli- 
gent person.®^ 

Interesting reader comments: 

Written in a superior tone that reads like a school teacher lecturing to kindcr- 

gartners. Attempt to write down takes away dignity. 

Classification of reader criticisms under adult approach: 

(i) Obvious appeal to ignorant, poor 
attempt at writing down 


B 3. LITERARY TECHNIQUES 

The specific methods that authors use in their endeavor to make 
their exposition clear, interesting, and vivid are classified under liter- 
ary techniques. The three ways in which authors are considered to 
do this are by using dramatic techniques, expository techniques, and 
figures of speech. 

B 3 a. DRAMATIC TECHNIQUES 

The term dramatic is used in a wide sense to include such methods 
of putting life into facts as stories, examples, comparisons, and con- 
trast. 

Importance quarter: 2 
Positive-negative ratio: 85-!- : 15 — 

Evidence from other experiments: 

Snyder [36] paired over three hundred junior high school children 
for intelligence and gave them two forms of the same material. The 
material was a lesson about money, one form written factually and 
the other dramatically. A test given immediately afterwards and re- 

“ See Appendix I A, B 2 a. 
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peated two months later showed that the knowledge of the two 
groups, reading different forms, was practically the same according to 
several different measures. At the end of the time each group was 
given the other form to read, to choose the more interesting one. 
Seventy-six per cent preferred the story. 

Exfert ofinions: 

In editing a school newspaper we have found that longer, more 
interesting presentations of news events are preferred to pithy, 
factual ones.*® A writer must study his group before choosing a 
technique. Stories often repel highly educated people, while they are 
necessary to gain the attention of the less educated.®^ 

Intersstmg reader comments: 

The typical example used was dramatic and effective. Quotation lends sup- 
port to convincing thought in conclusion. Story embellishment makes it 

saccharine. Devotes too much space to the story and not enough to facts. 

Classification of reader criticisms under dramatic techniques; 

(a) Examples 

(i) Vivid; typical; rich (i) Too many; illustrations 

with evidence irrelevant 

(b) Story method 

(i) Good way to depict condi- (i) Unreal; interferes with 

tions; makes facts easy necessary details 

(c) Comparison 

(i) Good (i) Too far-fetched 

B 3 b. EXPOSITORY TECHNIQUES 

Readers in this study requested more often that explanations be 
good than that they be dramatic, although both dramatic and ex- 
pository techniques ranked high in importance. In a few instances 
the negative criticisms called for dramatization, but they generally 
asked that statements be explained more fully. 

Criticisms of readers that express ease or difficulty of understanding 
due to the explanation, that deal with the interest of facts to the 
reader, or that describe the method of ^explanation employed, belong 
under expository techniques. 

Importance quarter: I 
Positive-negative ratio: 65-!- : 35 — 

” See Appendix I A, B i b. 

“ See Appendix I A, B i d. 
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Evidence from this study: 

After the reader had finished a series of pamphlets he was asked 
to rank all three in his order of preference for them, and then to 
rank them again in answer to the question, “Which meets your own 
needs as far as explanations go?” 

The agreement between explanations and order of preference was 
much greater than between preference and any other factor rated by 
them. In this study “good explanation” was proved to be the strong- 
est determinant of preference which was investigated.*® 

Classification of reader criticisms under expository techniques: 

(1) Easy to understand; difficult (i) Facts just stated, not 

subject simply explained explained 

(2) Vivid array of facts in (a) Plain statements; true but 

simple language dull; no trimmings 

(3) Well and clearly explained; (3) Badly explained; ambiguous; 

detailed not clear 

(4) Gives specific reasons; good (4) Merely recapitulation of 

review; answers one question foregoing 

well 

B 3 C. FIGURES OF SPEECH 

Comments upon figures of speech occurred rarely, but were of 
sufficient frequency to merit a separate category. 

In reviewing the section comments for a particular paragraph, the 
writer was interested to see that a figure of speech had been ap- 
plauded under five different labels. The important fact was not 
that the author had used a metaphor or hyperbole or parable, but 
that he had effectively caught the reader’s attention. 

Importance quarter: 4 
Positive-negative ratio’ fiy-j- : 33 — 

Classification of reader criticisms under figures of speech : 

(i) Good use of analogy, premise (i) Too many hyperboles, etc. 
and conclusions; of hyperbole, 
simile, metaphor, paradox 

B 4. FORMAT 

The reader’s comments on format were few but pointed. The 
subconscious influence of the appearance of pages, quality of paper, and 
so on, upon readers’ judgments may be greater than their infrequent 
comments would suggest. All criticisms of the style of the page, 
the footnotes, illustrations, etc., are grouped under format. 

“See Chap. V, p. 130, and Table XVIII. 
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Evidence -from this study: 

The experimental readers were asked to rank each series of three 
pamphlets on “the most attractive in the kind and size of print used 
and in the appearance of the page.” They agreed more closely 
among themselves about the order of ranking pamphlets on appear- 
ance than on any other of five qualities that they ranked. The re- 
lationship between the readers’ ranking of preference for each series 
and of appearance was negligible. 

Inspection of the four highest ranking and the four lowest rank- 
ing pamphlets in appearance led to the following generalizations: 
glossy paper and average size of type are definitely better liked than 
rough grayish paper and small type.^“ 

E 4 a. TYPE 

While many experiments have been made to discover the proper 
size and blackness of type, the experimental readers indicated by the 
infrequency of their criticisms that type was not important to them. 
Importance quarter: 3 
Positive-negative ratio: ly-j- : 83 — 

Evidence from other exferiments: 

In 1926, Pyke [40] summarized over two hundred studies in 
legibility which had been performed in England and America. In 
other words, a quantity of research on this topic has been done. 
Gates [37] believes that much of it has been fruitless because the 
investigators have set up artificial conditions which do not correspond 
with ordinary reading methods, and because they have not taken 
into account other causative factors. One important factor, he says, 
is that of habit: we read most easily what we have been accustomed 
to read. 

To test this hypothesis. Gates used two groups of children who 
were beginning to read. One group read lines of the usual length, 
100 to 102 millimeters, all year. The other read materials ranging 
in line length from 55 to 175 mm. At the end of the year the 
first group showed definitely greater speed on the 100 mm. lines; 
the second group could read lines of any length with the same speed. 

Hovde [38 1 set out to discover differences in rate of reading with 
different sizes of type and different leading (space between lines). 
The material was set up as a serial story, each incident separate, but 
all extracts from the interesting “We” by Lindbergh. The material 
See Chap. Ill, p. 45, and Appendix TV B. 
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was set up twice, the first time with variations in type and leading 
running from i to 16, the next time in reverse order, 16 to i. The 
surprising discovery was that “the order of the context far outweighs 
mechanical features in influencing reading rate, or legibility.” There 
was no perceptible order of eate from smallest to largest type. The 
readers said that they preferred larger type and wider leading, but 
their opinions rarely agreed with their fastest reading rates. 

In spite of these adverse findings, Paterson and Tinker [39] 
thought that there must be some outer limits to desirability in size 
of type and line length. They conducted wide experiments with 
tests standardized for the purpose, and reached these conclusions: 

10 point type was read somewhat faster than other sizes. 

Lines should be between 70 and 90 mm. in length. 

Both line length and type size should be balanced to have proper conditions 
for legibility. 8 point type in yz mm. line length and 10 point type in 
80 mm. line length were comparatively good. 


Evidence from this study: 

Each series of three pamphlets was ranked by the readers upon at- 
tractiveness of type.*'' The four most attractive pamphlets, those 
that ranked first in each series, and the four least attractive, ranking 
third, were examined to see whether the readers had a common stand- 
ard when they selected one class and rejected the other. The four 
most attractive pamphlets had heavier type and more leading than 
the four least attractive. Two of the least attractive pamphlets were 
ordinary in type. The other two had much smaller type than any of 
the other twelve pamphlets; one of these had insets and footnotes of 
still smaller type,*® 

Classification of reader criticisms under type: 


(i) Good use of italics 

(2) Type not clear 

(3) Type too small 

B 4 b. SIZE OF BOOK 

Under size of book are classified criticisms of the number of pages, 
of the area of a book, and of the book’s length in relation to the time 
that a person may spend in reading. Some experts who expressed 
opinions feel that one standard of exposition should be physical 
brevity of articles. 


Importance quarter; 3 
No comments in section reactions 


See Chap. V, Table XVII. “See Chap. V, p. 14.6, 
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Evidence from this study: 

When readers stated under snap judgments which pamphlet in a 
series they would like to read first, one of the reasons they gave for 
their choice was the size of the pamphlet,®® Four pamphlets were 
chosen by six readers in snap judgment because of their size; they 
were all a little smaller than average among the twelve, 

Exfert opinions: 

Brevity of statement is important so that the ordinary person may 
learn through short expositions of a great many subjects his con- 
tribution to and intrinsic place in modern life,® 

Classification of reader criticisms under size of book: 

(1) Interesting because long and (i) Bulky report unattractive 
detailed 

(2) Can be read in short time 

(3) Attractive because short 

B 4 C. COVER AND ILLUSTRATION 

Criticisms by readers of the various “art features” are grouped to- 
gether under cover and illustrations. 

Importance quarter: ^ 

2 positive comments, 2 negative 

Evidence from other experiments: 

Mellinger [46] found that children of three ages (six, eight, ten) 
preferred realistic colored pictures to conventionalized black and 
white ones. 

Evidence from this study: 

“Attractive cover” ranked fourth as a cause for snap judgment 
preference.'^^ Of the two most often mentioned pamphlets the 
readers said: 

K — Intriguing; print and cover attractive; cartoon (on cover) impressive and 
forceful; make-up attracts attention. 

F — ^Attractive layout of cover; neat; colorful and appealing. 

Expert opimons: 

Far more attention should be paid to choice and reproduction of 
pictures. Illustrative material has been all but overlooked for edu- 
cational purposes.'^® 

See Appendix I B. 

" See Appendix I A, B 3 a. 

“ See Appendix I B. 

“ See Appendix I A, E 3. 



STANDARDS FOR SELECTING MATERIAL 187 
Classification of reader criticisms under cover and illustrations: 

(1) Colorful 

(2) Cartoon impressive, forceful (2) Need pictures to clarify 

explanation 

B 4 d. FOOTNOTES, INDEX, AND TABLE OF CONTENTS 

The footnotes, index, and table of contents were evidently not 
widely used by the experimental group. These qualities of format 
would doubtless hold more importance for readers using materials 
as references or textbooks. 

Importance quarter: 4 
Positive-negative ratio: 33+ : 67— 

Evidence from this study: 

Two readers selected books to read first because they had a 
“promising table of contents.”*® 

Classification of reader criticisms under footnotes, index, etc, 

(All of the section and general comments concerned footnotes) 

(l) Well footnoted for the (l) Footnotes kill interest; over- 

serious reader use of notes and quotes 

B 4 e. SECTIONS AND CAPTIONS 

Most textbooks, as well as pamphlets, are divided into sub-chap- 
ters or sections. Any reader criticisms of the section divisions or 
of their captions are classified under 640. 

Importance quarter: 4 
2 positive comments, 2 negative 

Interesting reader comments: 

Titles for each section anticipate not only subject, but purpose of discuss- 
ing it. Dynamic titles. Sections should have been joined, for all deal 
with same thought. Sections much too small, too dependent on each other. 
Classification of reader criticisms under sections and captions: 

( I ) Section captions interesting 

(2) Sections too small 

B 4 f. TITLE OF BOOK 

The title was the major influence upon the experimental readers’ 
preliminary selection of a book. Authors and editors recognize the 
importance of good titles, as evidence of other experiments shows. 
Importance quarter: 3 
2 negative comments 


See Appendix I B. 
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Evidence from other exferiments: 

The most dramatic report of the influence of the title is given 
by Haldeman-JuJius [48]. He is the editor of “The Little Blue 
Books” which are bought from a check list of titles. The title is 
the book’s only advertisement. He gives many stories of books that 
had practically no sale till the title was “pepped up”. He says that 
key words for titles are “The Truth about . . “Life . . “How 
to . . .”. Descriptive phrases should follow names in biographies, as 
“Robert Ingersoll, Benevolent Agnostic”. People are generally 
more interested in ideas than in personalities, according to his ex- 
perience. 

Hart [49] analyzed ten best selling novels and concluded that “the 
fiction reader likes best, apparently, those titles which arouse curiosity 
without at once revealing their significance. The title should be 
closely related to the theme of the novel, however, if it is to satisfy 
the reader.” 

Brome [47] quoted the interest of several authors in titles. Max- 
well thought his best title to be “Spinster of this Parish” because 
“the phrase is familiar and flows easily, it suggests the compass of 
the story in four words, none of which is clumsy.” 

Evidence from this study: 

The title led in importance for determining snap judgment prefer- 
ences.^* Some comments on the most often mentioned titles were: 

“Unemployment and its Remedies” — Seems to suggest that remedies for 
situation are given. Hits the problem on the head. Suggests a discussion of 
current evils. Suggests remedies, not reliefs. Sounds constructive. 

“Inflation and Your Money” (sub-title: “A story of money that everyone 
can understand”) — It purports to be a story. Your money is provocative. 
Title promises an understandable history of a subject that is rarely presented 
simply. From the title I am led to believe that I may be able to see the con- 
nection between my money and inflation. 

“Fight Against War” — Challenging. Carries force. Catches the eye. 
Sounds emotional and not abstract. 

Classification of reader criticisms under title of book; 

(l) Challenging, constructive; 
hits an important problem 

(2) Slang in title looks bad 

(3) Title is misleading 

(4) Not appealing 

“ See Appendix I B. 
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B 4 g- MAKE-UP OF BOOK 

Under make-up of book quality of paper, binding and any other 
general criticism of the make-up should be classified. 

Importance quarter: 4 
No section comments 

Evidence from other exferiments: 

The make-up of books is a technical question on which printers 
have assembled the best experimental evidence. The Technical 
Association of the Pulp and Paper Industry has issued and kept up- 
to-date the “Tappi Standards,” “a loose leaf manual of tentative and 
official testing practices that have received recognition by the Tech- 
nical Association.” [43] This association assembles scientifically 
tested information on permanence and durability of paper, qualities 
of different types of binding, and other qualities of make-up. 

Evidence from this study: 

Under snap judgments pamphlets P and Q were each preferred 
once because they were “well made books.”^® 

Classification of reader cnucisms under make-up of book. 

(i) Well made book (i) Poorly made book 

C. CATCHING AND HOLDING READER INTEREST 

After careful deliberation, catching and holding reader interest 
was given a separate classification. It overlaps somewhat with other 
standards and, in some respects, is more generalized than the ma- 
jority of them. An analysis of the methods under this standard, 
however, should show more clearly than methods under other stand- 
ards how to obtain the qualities that make satisfactory reading. It, 
with its subdivisions, was therefore drawn apart for examination 
and definition. 

The standards that occur under this topic are: 

C. Catching and holding reader interest 

1. Introduction 

a. Outline of plan 

b. Awaking interest 

2. Conclusion 

a. Summary 

b. Outcome of reader interest 

3. Techniques for relating subject to reader 

4. Emotional appeal 

5. Use of famous author's name 

“ See Appendix I B.' 
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C I. INTRODUCTION 

In the classification the term introduction generally applies merely 
to the first section or chapter. Occasionally there are books with 
such definite changes of content that interest must be aroused anew 
in the middle 5 then comments in the body of the book about the 
introduction of new topics would also be included under introduction. 
For example, in one pamphlet, where three different economic sys- 
tems were described, the few remarks at the beginning of each new 
description were classed as introductory. 

Many of the criticisms of the introduction are not classified here, 
for they pertain to one of the other standards of judgment. Only 
the criticisms that tell how the introduction is made or that cite 
methods used by the author to awaken interest are included under 
introduction. 

Evidence from this study: 

On the whole, introductions seemed to be popular. Six of the 
twelve were among the best liked sixteen per cent of sections read. 
Only one of the introductions was the least popular section in its 
pamphlet. Two were among the two most popular in their booklets. 
In other words, two-thirds of the introductions were very well liked. 

C I a. OUTLINE OF PLAN 

Criticisms aroused by the author’s method of indicating the pro- 
posed content of the pamphlets are included under outline of plan. 
Importance quarter: 4 
Positive-negative ratio. 88-j- : 12 — 

Interesting reader comments: 

Goes right to heart of the subject, thus arousing reader interest. Slow in get- 
ting to point. Plan and object of book fuEy discussed. 

Classification of reader criticisms under outline of plan: 

(1) Definite statement of plan (i) Purpose remains obscure 

and purpose 

(2) Gives setting for follovvlng; 
raises questions to be answered 

C I b. AWAKENING INTEREST 

The techniques used by the author to awaken interest brought 
more comments than did his method of introducing the subject. 
Importance quarter: 2 
Positive-negative ratio: 76-f- ; 24 — 
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Interesting reader comments: 

Zestfulness of beginning makes reader want to go on. Relationship to reader 
at once dwelt upon. 

Classification of reader criticisms under awakening interest: 

(1) Simple explanation of hard (i) Technical; stress on diffi- 

snbject cnlty is forbidding 

(2) Promises interesting discus- 
sion; relation to reader 
stated; provokes curiosity 

(3) Clear-cut; catchy, enthusiastic (3) Weak; bad; not alive 

(4) Easy stages; directly to the (4) Too abrupt; too slow; 

point too hurried; too long 

C 2. CONCLUSION 

Although conclusion refers in most cases to the last portion of 
the book, the terra may sometimes be broadened to include the end 
of sections or chapters, if the summaries there mark a definite period 
to a train of thought. Only those criticisms that apply specifically 
to conclusions and that can be classified under summary or outcome 
of interest belong under C 2. 

Evidence from this study: 

Conclusions were not as popular as introductions. Five of the 
twelve were among the sixteen per cent of most popular sections. 
Only three were the most popular sections in their respective pam- 
phlets. Three were among the sixteen per cent least popular. One 
was the least liked section in its pamphlet. 

C 2 a. SUMMARY OF BOOK 

Classified under summary are remarks concerning the author’s 
method of concluding his pamphlet, opinions about effective devices 
for summarizing, and criticisms of the quality of the summary. 
Importance quarter: 3 
Positive-negative ratio: 67-I- : 33 — 

Classification of reader criticisms under summary of book: 

(1) Summary comprehensive; (i) Questions not answered 

detailed; short 

(2) Ties all loose ends to (2) Not unified; does not 

original thought summarize 

(3) Strong (3) Weak 

(4) Review directly involves 
reader 
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C 2 b. OUTCOME OF READER INTEREST 

The success of reading material may be judged to some extent by 
its after-effects. These experimental pamphlets were selected with 
the hope that they might be introductory to more extended reading 
for some of the experimental group. It was therefore interesting 
to note, in the first place, whether readers felt they had obtained any 
new ideas or suggestions from the pamphlets and, in the second place, 
whether any readers expressed an intention to follow the subject 
with further thought, study, or action. 

Importance quarter: 2 
Positive-negative ratio’ 79-I- .21 — 

Interesting reader comments: 

(All of these comments are taken from the general criticisms, at which point 
this standard is applied more fully than in the section reactions.) 

Questions not answered. No solution given. Does not clear up questionable 
statements. Provides much rational food for thought. Should be subject of 
a debate. Interesting because of proposed method for accomplishing hoped- 
for end. A good introduction to subject. No enlightenment on topic. Not 
one practical suggestion made for solution of immediate national problems. 
Quo vadis^ 

Classification of reader criticisms under outcome of reader in- 
terest: 

(1) Good solution, raises new (1) Problems not solved 

questions 

(2) Predicts (2) Doesn’t explain solutions 

(3) Suggests further thought (3) Too much like home-work 

and study 

(4) Practical (4) Impractical; hypothetical 

(5) Constructive appeal to 


C 3 . TECHNIQUES FOR RELATING SUBJECT TO READER 

Even though the subject of a treatise be of great Importance, spe- 
cial means should be employed by authors to insure readers’ appre- 
ciation of it. Authors may catch the attention of readers by telling 
stories, using italics, giving clear explanations, or making wise use 
of any other of the thirty-eight qualities that readers demand in their 
reading. The authors of the experimental pamphlets used fourteen 
different techniques for relating their subjects to the reader. 

The only criticisms which should be grouped under techniques for 
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relating subject to reader are the readers’ responses to the use of 
such techniques. 

Importance quarter: i 
Positive-negative ratio: 59-|- : 41 — 

Evidence from this study: 

Several assistants in the experiment read each pamphlet carefully 
in order to determine what methods the author used “to catch and 
to hold the reader’s interest.” They found that several methods were 
employed in each pamphlet. All of the judgments of the assistants 
were synthesized by the writer into an analysis of the so-called 
“human interest” techniques. The analysis is listed for each pam- 
phlet in Appendix III D. In addition to citing the methods in re- 
lation to pamphlets, the writer classified the methods under the 
standards in the Combined List of Criteria. 

The list of the techniques for relating subject to reader that were 
used in the experimental pamphlets follows, with letter and numeral 
referring to the position of the technique in the Combined List of 
Criteria; 

A I, 3 — Supply of background and of facts. 

2 — Facts in graphic form. 

B I — Simple language. Explicit definitions. Generalizing topic sentences, 
elaborated with concrete examples. 

2 — Colloquial language. Conversational style. Colorful description. 

3 a — Story of a typical family. Comparison to show good and weak points, 

similarities and differences. Direct quotations in dialogue form, 
b — Selection of important and effective thought, 
c — Striking analogy in introduction. 

4 — Italics to emphasize important facts. Cartoon. Captions that suggest 

the thought. Content in quotation marks. 

Q 2 — Frequent summaries. 

4 — Account of suffering, loss of life. Use of emotional writing at stra- 

tegic points. Ridicule. 

D I — ^Author an expert, with a calm, scholarly attitude. 

2 — Unbiassed arguments on both sides of the issue. 

Special techniques that could only be grouped under C 3' 

C 3 — Questions that relate reader’s experience to problem under discussion. 

Emphasis on universality of consequences. 

Interesting reader comments: 

Could imagine myself one of the characters in the story. Fails to humanize 
facts presented. Points to direct relation with “me.” Hits vital question in 
my mind. 
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Classification of reader criticisms under techniques for relating 
SUBJECT TO reader: 

(1) Problems vital to reader 

(2) Direct appeal to reader; 
question directed at him; 
questions and examples of 
human interest 

(3) Problem lives; personal style 
C 4. EMOTIONAL APPEAL 

Most of the criticisms that mention the author’s emotion are easily 
identified under this heading. Although the “calling of names” is 
generally classified under emotional appeal, some descriptions of the 
author are placed under influence on reader, and a few under in- 
formation. That is, if the opinion reads, “The author is a liar,” 
it is placed under emotional appeal} if it says, “The data presented 
are contrary to known facts,” it is placed under information. “The 
author does not convince me of his sincerity” would be placed under 
influence on reader. In the main, then, emotional reactions toward 
the author and his book are grouped under emotional appeal. 
Importance quarter: i 
Positive-negative ratio. 67+ : 33 — 

Evidence from this study: 

Although the section reactions in the above ratio showed a balance 
of positive commendations of forms of emotional appeal, the general 
criticisms showed an opposite tendency. Since a detailed classifica- 
tion of general criticisms was not kept, the exact proportion of positive 
to negative criticisms was not known; but it was over fifty per cent 
negative. The interpretation of this fact may be that, although cer- 
tain injections of emotion may be liked in reading, a definitely emo- 
tional pamphlet arouses more unfavorable than favorable comments. 

Interesting reader comments: 

Author shows ability to communicate deeply rooted emotions. A lofty appeal, 

but lacks force and substance. Author is trying to browbeat his own in- 
stincts into reader. Strong emotional tone gives color and warmth to context. 

Classification of reader criticisms under emotional appeal: 

(1) Author keeps discussion (i) Not much enthusiasm shown; 

controlled; without heat facts cold; no enthusiasm 

(2) Appeal is heroic, patriotic, (2) Appeal is a preachment, too 


(1) Reader not interested in 
subject 

(2) No conscious appeal to 
reader; no human interest 


(3) Too academic; too realistic 
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convincing, stirring, elo- idealistic, too depressing, too 

quent, idealistic enthusiastic, too bitter, 

(3) Author is honest, sincere, sarcastic 

shows a keen sense of humor 

(4) Especially good ethics; uni- 
versal appeal of brotherhood; 
courageous expression of an 
ideal; reader made to feel 
situation 

(5) Enthusiasm begins to wane 

C 5. USE OF FAMOUS AUTHOR’S NAME 

Undoubtedly one of the editorial feats of showmanship is the 
display of a famous author. Editors may wonder how successful 
they have been in sustaining the interest they have aroused by pub- 
lishing a book by a famous author} the success may be discovered by 
obtaining the criticisms of readers. In this study the use of a famous 
name aroused great interest in one pamphlet, but the interest was un- 
justified by the reading. 

All criticisms that indicate a knowledge of the author’s identity 
belong under famous author. 

Importance quarter: 3 

Only 2 comments, both positive 

Evidence from this study: 

Pamphlet F, by Einstein, was selected to be read first in its series 
far oftener than any other pamphlet. Seventy per cent placed it 
first under snap judgments. Only thirty-nine per cent liked it best of 
the three pamphlets after reading it.^® The remarks of the famous 
author were placed in a favorable setting by the editor, who prefaced 
each selection with a note, a story, or a historical allusion} but other 
qualities than the famous author operated to defeat the popularity 
of the book. 

“Familiar author” ranked second in snap judgments. Nineteen 
readers selected pamphlet F for this reason. Some of them said : 

Acquainted with author’s fairness 

Curious about famous man’s opinions on this subject 

Aware of author’s earnestness and sincerity 

Author IS acknowledged genius 

Has international reputation for knowledge of subject 

His opinions well worth reading 

"See Chap. V, p. 127. 
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Classl-fication of reader criticisms under use of famous author’s 
name: 

(Criticisms of the famous authors occurred so rarely tliat this standard should 

be eliminated as a major classification) 

(l) Interesting analysis of famous 
author’s feelings and sentiments 
D. DEALING WITH CONTROVERSIAL ISSUES 

The standards that are organized under dealing with controversial 
issues need not receive so much space in lists of standards which are 
not intended to be applied to current problem literature. They are 
important, however, in a wide area of study group material. The 
following standards are classified under D; 

D. Techniques of controversy 

1. Author’s display of his own viewpoint 

a. Non-partisan 

b. Fair partisan 

c. Prejudiced partisan 

2. Techniques of controversy 

a. Presentation of argument 

b. Criticism 

c. Influence on reader 

D I. author’s display of his own viewpoint 

In Chapter V (page 123) four possible types of an author’s bias 
are described. Three of the four types of bias are used in the classi- 
fication under a, b, and c below. The fourth type of bias, which was 
there described as the “salesman’s view,” i.e., an author who pre- 
sents his own belief without referring to alternative views, may be 
variously classed under or c as follows. If the reader’s criticism 
says that the author’s purpose is to instruct upon a subject that al- 
ready has majority agreement, his criticism may be classified under 
fair partisan. If, on the other hand, the author gives the reader an 
impression that his is the only possible opinion, even though it is 
not generally accepted, or if the reader indicts him for propagandiz- 
ing instead of educating, the criticism is classified with prejudiced 
partisan. 

Evidence from this study: 

In Chapter V, Table XXI, page 136, descriptions of bias are pre- 
sented in order to compare the author’s display of viewpoint with 
the reader’s preference for pamphlets. Although those pamphlets 
which were described by assistants as prejudiced were on the whole 
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most popular, the readers disliked pamphlets written by authors who 
sponsored a cause without indicating any alternative beliefs. 

D I a. NON-PARTISAN 

The writer who is non-partisan examines all viewpoints without 
displaying a brief for any one of them, according to the definition 
in connection with the investigation of bias mentioned above. Criti- 
cisms by readers that indicate an author’s impartiality are classified 
here. 

Importance quarter: 2 
Positive-negative ratio 67-!- ‘33 — 

Exfert of inions: 

The writer should give fair presentation to different points of 
view.*^ Current problems should be treated as non-propaganda when 
writing curriculum materials for adults.^'’ 

Classification of reader criticisms under non-partisan: 

(i) Maintains impartiality, (i) No definite attitude; author 

open minded fails to register point of view 

D I b. FAIR PARTISAN 

In order to be considered a fair partisan the writer admits a bias for 
one side of the controversy. He treats the other side fairly, stating 
both its strong and its weak points. When the readers recognize 
this quality in an author, their comments are classified under fair 
partisan. 

Importance quarter: 4. 

3 positive, I negative comment 
Interesting reader comment: 

Leans over to one side, but makes fair presentation. 

Classification of reader criticisms under fair partisan: 

(i) Definite idea of how author (1) Information that only half 

feels; shows generosity, is fair informs is dangerous 

Die. PREJUDICED PARTISAN 

When the author is called a prejudiced partisan he may ignore 
other viewpoints completely, stating only the advantages of his posi- 
tion. Or he may show only the fallacies of other positions and only 
the strength of his own. Criticisms which challenge the author’s 
prejudice are grouped under this standard. 

See Appendix I A, D z. 
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Importance quarter: 3 
Positive-negative ratio: ii-f- : 89 — 

Interesting reader comments: 

Author’s purpose is to influence, not to instruct. Bias of author so evident 
that it is not interesting; reader not drawn to sympathize with author’s views. 
Sounds like patent-medicine ad, yet interesting. 

Classification of reader criticisms under prejudiced partisan: 

(l) Interesting “medicine adver- (i) Too biassed; propaganda; 

tisement” style point of view one-sided 

D 2. TECHNIQUES OF CONTROVERSY 

All of the readers’ criticisms which were classified above under 
author’s viewpoint related only to his display of his own views. His 
general method of dealing with controversial subjects was of much 
greater importance to the experimental readers, 

D 2 a. PRESENTATION OF ARGUMENT 

When the reader criticizes the presentation of the argument, his 
own point of view sometimes prevents him from seeing clearly the 
author’s intent in the argument. Under this heading reader opinions 
about the methods that the author used in argument are grouped. 
Importance quarter: 2 
Positive-negative ratio: 54+ : 46 — 

Interesting reader comments: 

Author is muddled. Argumentation quite convincing regardless of emotional 
prejudice. Good scientific arguments to uphold author’s contentions. Gives 
no reasons for conclusions. No stress is laid on important points. By inter- 
esting analysis derives a logical answer. 

Classification of reader criticisms wider presentation of argu- 
ment: 

(1) Point by point examination (i) Statements doubtful; author 

of opposing arguments misinterprets; disregards 

truth 

(2) Logical (2) lUogical 

(3) Intelligent; good reasoning (3) Wrong emphasis; examples 

do not prove 

D 2 b. criticisms 

The readers’ remarks about the author’s manner of analysing 
other points of view are catalogued under criticism. 
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Importance quarter: 4 
Positive-negative ratio: 86-}- : 14 — 

Classification of reader criticisms under criticism : 

(1) Absence of destructive criti- 
cism and caustic manner 

(2) Is just, clear, fair, forceful, (2) Is unjust, merely vindictive 

comprehensive, constructive abuse 

in criticism 

D 2 C. INFLUENCE ON READER 

The esssence of the standard, influence on reader, lies in the 
reader’s expressing himself as convinced of, doubtful about, or op- 
posed to the suggestions embodied in the pamphlet. 

Importance quarter: 2 
Positive-negative ratio: 55-I-. 45 — 

Interesting reader comments: 

Too idealistic and Utopian. Leaves reader with a blank feeling. Not 
dramatic enough to appeal to one not of author’s opinion. 

Classification of reader criticisms under influence on reader: 

(1) Good advice; difficult subject (i) Reader disagrees with 

well treated author 

(2) An appeal to reason 

(3) Convincing (3) Not convincing; creates 

doubt 
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APPENDIX I A-i 

CLASSIFICATION OF OPINIONS EXPRESSED BY EXPERTS 
IN BOOKS AND INTERVIEWS CONCERNING STANDARDS THAT 
READERS USE IN SELECTING READING MAITER 

The following opinions about the standards for selecting reading matter were 
accumulated during interviews with adult educators, librarians and editors and 
in the course of reading many books and articles. Standards were tabulated 
from twenty-five sources. In cases where the standards were merely listed or 
mentioned without discussion, no reference is cited below. Listings are counted, 
however, in the frequency of the summary table (A-z)'. 

Opinions about the determinants of the readability of books were not ob- 
tained in more detail because William S. Gray has already canvassed the field 
of opinion in an exhaustive questionnaire addressed to ninety librarians, editors, 
and adult educators.^ 

Non-fiction rcadmg maUrtd for adults should — 

A. Recognize differing needs of readers: 

I. Should touch on basic interests of all readers. C 3* 

By an abiding human interest I mean this: there are some things that 
everyone has in common with every other one. The man who makes 
our literature seizes upon the things that are universal, so that when 
the great product is put before you, you see that it appeals to you as 
if it were spoken or written for you alone.^ 
or 2. Should be of special interest to one or more groups. C 3 

We have found in the Y.W.C.A. that there must be standards for 
special groups rather than for all adults en masse. We differentiate on 
an occupational level.®' 

^ Gray, William S. and Leary, Bernice. What Makes a Book Readable, Univer- 
sity of Chicago, Chicago, III., 1935, Chap. II. 

* Letters and numbers in italics refer to classification under the Combined List of 
Criteria. 

"Hendrick, Burton J. Vise Training of an American (the earlier life and letters 
of Walter Hines Page), Houghton MifiSin, New York, igaS, p. 287. 

“Coyle, Grace, Laboratory Division, Y.W.C.A. Interview, June, 1928. 
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Non-fiction leading material for adults shotdd — 

A. Recognize differing needs of readers (Cont.) : 

3, Should be based on and intimately tied up with readers’ immediate inter- 

ests. _ _ _ _ _ Asa 

. . . current topics insistently clamoring for attention.^ 

4. Should perceive differences in concepts, distinct from vocabulary diffi- 
culties. Bib 

A German scientist m Darwin’s time literally could not conceive of 
evolution. I talk to many groups that cannot comprehend the facts 
and demands of the concept of social change. Mary Van Kleek doubts 
whether the individualistic middle class will ever be able to under- 
stand the idea of interdependence of the workers, let alone guide them 
or write for them.® 

B. Aim to catch and hold the reader’s interest. 

1. By dramatic techniques. B 3 a 

a. A less highbrow magazine for parents than the present ones would 
put across its points by fiction, by representation of simple case stories, 
with pictures and records of group activities.® 

b. The magazine. Current Events, has found that longer, more interesting 
presentations of news events are preferred to pithy factual ones.® 

c. Business girls are using the modern novel to study sociological and 
economic problems.® 

d. Study your group when selecting a technique; stories often repel the 
upper intellectual levels while attracting the lower.® 

2. By recognizing the reader as a layman. B 2 f, B 3 b 

a. Talk not down but across to him. You merely have a grasp of one 
small corner of knowledge which he does not possess; treat him as a 
being of equal intelligence, possibly superior to you in other fields.'^ 

b. It is important to recognize that it is not sufficient just to state the 
problem. Time must be taken for a description of the situation as it 
appears to various members of the group. Each person must under- 
stand and feel the problem as it looks to others.® 

c. The treatment should be non-academic. 

‘ Hendrick, cit , p. 207. 

' Miller, Marion, Education secretary. Child Study Association. Interview, June, 
1934. 

* Sayre, Harrison B., editor of My Weekly Reader, Youth’s Comf anion, Current 
Events, etc. Interview, Nov., 1933. 

’ Cowles, Imogene, English instructor, Columbia University. Interview, Oct., 
1933 - 

“Elliott, Harrison S. The Process of Group Thinking, Association Press, New 
York, 1928, pp. 42-3. 
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Non- fiction reading material for adults should — 

B. Aim to catch and hold the reader’s interest (Cent.) : 

3. By sensing the reasons why he comes to the book for help. B 4 by A a 

a. Brevity of statement is important so that the ordinary person may learn 
through short expositions of a great many subjects his contribution to 
and intrinsic place in modern life.® 

b. People want facts rather than theories when they are looking for help 
with problems.^® 

but c. Lists of statistics are discouraging. Avoid overweighting with compli- 
cated graphs and detail. Statistics should be supplemented by simple 
diagrams.^^ 

4. By special literary techniques. B 3 a 

a. Suspended interest. 

b. Take as concrete forms of the great problem as you can find. . . . 
Suppose you discover at these places life stories and human experiences 
which illustrate in a striking way the lift from the old darkness of 
slavery to the ambitions of life of American citizenship. Would not 
such definite experiences as these, if properly interpreted, really illu- 
minate the whole subject in a way that no amount of theory or educa- 
tional discussion or didactic commonplace ever illumine iti^® 

C. 

1. Of rhythm and fluency. Bad 

In writing for immature people there must be short sentences. But at 
the same time I try to get flow of thought from one sentence to an- 
other. Rhythm is the most essential feature in writing for children.^® 

2. Of terse, simple language. B a e 

And when it came to style — directness, simplicity. Page loved one or 
two syllabled Anglo-Saxon words. . . . “There is no vehicle that can 
carry great literature except those plain words that roll out of our 
mouths when we suffer great emotions, when we weep, when we pray, 
when we laugh. Those words we speak in the earnest moments of our 
lives, those idioms that we use then — ^they are the clothing that takes 

’’American Library Association. Libraries and Adult Education, Macmillan, 
New York, igafi, p. 54. 

“Flexner, Jennie M., Reader’s Adviser, New York Public Library. Interview, 
June, 1934. 

“Ridley, C. E. “Annual Appraisal of Municipal Reports.” National Municifal 
Review, 1934, 23.14-16. 

’“Hendrick, of. At., p. 210. 

“ Rugg, Harold O., Author, professor of education, Columbia University. Inter- 
view, May, 1934- 
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t^onr-jiction reading material for adults should — 

C. Possess literary quality (Cont.) : 

the common thought and in the hand of a great master immortalizes 
it. Keep close, close, close to the idiom!”^'* 

Eighty-six and two-fifths per cent of all the world’s fights and argu- 
ments occur because the contestants have failed to define the words 
over which they clash. The great tragedy of violence is that when 
the dead and wounded are counted it will often be discovered that 
many of the slain need not have died at all if only they had under- 
stood the other fellow’s language. What this nation and every other 
needs is a good collection of five-cent words. Half-dollar and two- 
bit words make for misunderstanding and strife.^® 

3. Of conciseness. B 2 e 

Page was the enemy of the diffuse. He would say, ''Do you realize 
that the story of the creation of the world, the biggest thing that ever 
took place, was told in a paragraph?”’-^ 

4. Of vitality. B 2 c 

Finally there is this most essential quality — ^vitality. . . . The books 
that will hold the attention of the casual reader of limited education 
will reflect the enthusiasm of the author in his subject and will bear 
evidence of his understanding of the psychology of his reader. It 
will be addressed to a flesh and blood reader; it will be something more 
than an exposition of the subject. . . 

D. Possess soundness and educative merit: 

1 . Should have intrinsic value. Age 

2. Should give a fair representation to different points of view. Should be 

non-propaganda. D z a 

3. Should leave unsettled points to 'work on; an introductory treatise should 

never create the impression of having covered the field.® C 2 b 

E. Appear in attractive format: 

1 . Booklet should be divided by captions into sections. B ^ e 

2, Attention should be paid to the cover, type, title, paper, and introduction 

to make people want to read farther.^' B 4 a, B 4 c 

Bulky, mimeographed books are unattractive. There should be a change 
of type for emphasis.^"^ 

“Hendrick, of, cit., pp. a 10, 113. 

“Broun, Heywood. “It Seems to Me” (syndicated), /few York Herald Tribune, 

April 26, 1934. 

“Felsenthal, Emma. “First Books in Many Subjects,” Journal of Adult Edu- 
cation, 1:203-225, 1929, p. 210. 

’■''Phillips, Edna, Librarian. Interview, Jan., 1934. 
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Non-fiction reading matetial for adults should — 

3. Far more attention should be paid to choice and reproduction of pictures. 
Illustrated reading material for educational purposes has been overlooked.^® 

B 4 = 

F. Give attention to the difficulty of reading material: 

1. Should be concrete. B i c 

2. Should be -written for different comprehension levels. Bib 

3. Difficult words should be defined and illustrated. B i a, B i i 

Angelo Patri’s sound, simple, anecdotal, perseveringly repetitious, always 
human work is an example of what I mean. . . . Through such 
constant presentation of the subject, exact technical terms of the milder 
sort are creeping into ordinary vocabularies ; and a wider knowledge of 
exact nomenclature always makes it easier to present accurate technical 
information.^® 

4. Should be non-technical, although “occasional technical words are liked, 

if they are made meaningful.”® Bio 

5. Paragraphs should frequently be short, since people tend to be headline 

readers. Bid 

“Hanna, Paul R., Editor of Building America. lntervie<w, Feb., 1934. 

” Fisher, Dorothy Canfield. Why Stof Learning? Harcourt, Brace, Ne-w York, 
1927, p. 144. 
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APPENDIX I A-2 

FREQUENCY OF EXPERT OPINIONS CONCERNING READING 
STANDARDS CLASSIFIED UNDER THE COMBINED 
LIST OF CRITERIA 


List of Criteria 

Expert Opinion 
Qassification 

Frequency 
of Expert 
Opinion 

quency 

Rank 

A. Content 

I. Historical allusions 


0 

30 

2. Presentation of figures 

B3b, B3C 

4 

6 

3. Treatment of subject 

a. Timeliness 

A3 

2 

IS 

b. Selection of information 


0 

30 

c. Quality of information 

Di 

I 

20 

B. Form 

I. Difficulty 

a, Terms 

... F 3 ,F 4 

4 

6 

b. Concepts 

— A4, F2 

8 

2 

c. Abstractness 

Fi 

2 

IS 

d. Condensation 

... Fs, F6 

3 

9-5 

2. Literary quality 

a. Organization 


0 

30 

b. Form and grammar 

... 

0 

30 

c. Vitality 

C4 

2 

IS 

d. Rhythm and fluency 

Cr 

3 

9.S 

e. Concise, simple language 

... C2, C3 

2 

IS 

f. Adult approach 

Baa 

2 

IS 

3. Literary techniques 

a. Dramatic techniques 

. . . . Bly 1 ^ 4 ^ 

6 

3 .S 

b. Expository techniques 

Bzb, Bac, B4b 

3 

9-5 

c. Figures of speech 



0 

30 

4, Format 

a- Type 

Ea 

z 

IS 

b. Size of book 

. , . B3a, Ez 

6 

3 .S 

c. Cover and illustrations 

... E 2 ,E 3 

4 

6 

d. Footnotes, etc 

... 

0 

30 

e. Sections and captions 

Ei 

I 


f. Title of book 

E2 

I 

20 

g. Make-up of book 

... 

0 

30 

C. Catching and holding interest 

I. Introduction 

a. Outline of plan 



0 

30 

b. Awakening interest 



0 

30 

2. Conclusion 

a. Summary of book 



0 

30 

b. Outcome of reader interest . . , 

Ds 


IS 




APPENDIX 

APPENDIX I A-2 (Contmued) 


Combined 

List of Criteria 

Criteria 

Expert Opinion 
ClaxsiHiJition * 

Frefluency 
of Expert 
Opinion 

Frc- 

3. Relating subject to reader 

. . Ar, A2 

9 

r 

4. Emotional appeal 


0 

30 

S. Use of famous author’s name , 


0 

30 

D. Dealing with controversial issues 

I. Author’s display of viewpoint 

a. Non-partisan 

Dz 

3 

9-S 

b. Fair partisan 


0 

30 

c. Prejudiced partisan 

, , , 

0 

30 

2. Techniques of controversy 

a. Presentation of argument . . . 


0 

30 

b. Criticism 

. . 

0 

30 

c Influence on reader 


0 

30 


* The classificalion refers to the outline form of the listing of entena m Appendix I A-z 
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APPENDIX I B 

REASONS FOR SNAP JUDGMENT PREFERENCE AMONG 
TWELVE PAMPHLETS 


Reason 

Frequency 

Classifi- 
cation * 

Frequency 
in Classifica- 

Rank in 
Classifi- 

Good title 

42 

B 4 f 

42 

j 

Interesting topic or content 

38 

A 3 b 

38 

3 

Familiar author 

35 

C 5 

40 

2 

Attractive cover 

22 

B 4 c 

23 

4 

Related to reader interest 

IS 

C3 

IS 

6 

Comprehensive; survey of field . 

14 

A 3 c 

19 

S 

Related to current problems 

13 

A 3 a 

13 

7 

Interesting style 

9 

B 2 c 

9 

S 

Simple 

8 

B I b 

8 

9 

Size of pamphlet 

6 

B 4 b 

6 

10 

Instructional; informational 

S 

A 3 c 



Identification with series or pub- > 
Usher 

5 

C 5 



Appearance of impartiality 

4 

D I 

4 

II. s 

Apparent clarification of problem . 

4 

B 3 b 

4 

II'S 

Good table of contents 

2 

B 4 d 

2 

13-5 

Well made book 

2 

B 4 g 

2 

13. S 

Good type 

I 

B 4 a 

1 

16 

Patriotic 

I 

C 4 

1 

16 

Attractive frontispiece 

I 

B 4 c 

.. 

.. 

Good chapter and section divisions 

* 

B 4 e 

* 

16 


* Indicates classidcation in the Combined List of Criteria. 
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APPENDIX I C 

COMPARISON OF RANKING OF STANDARDS IN COMBINED LIST 
OF CRITERIA ACCORDING TO GENERAL AND SECTION 
CRITICISMS OF TWELVE PAMPHLETS 


Combined 

I-ist of Criterion 

Criteria 

Frequency 

1 

Rank 

Section 

Frequency 

Rank 

A. Content 

I. Historical allusions 

.. x6 

17.5 

18 

20 s 

2. Presentation of figures 

. S6 

s 

35 

10 

3. Treatment of subject 

a. Timeliness 

■ • S 

27.5 

It 

26 

b. Selection of information . . . 

. . 25 

14 

58 

2 

c. Quality of information 

.. 87 


38 

9 

B. Form 

I. Difficulty 

a. Terms 

4 

30 s 

19 

19 

b. Concepts 

• • 94 

29 

16 

c, Abstractness 

• • 4 

30. s 

22 

r8 

d. Condensation 

.. 52 

7 S 

40 

8 

2. Literary quality 

a. Organization .• 

■ S2 

7-S 

30 

IS 

b. Form and grammar 

• 19 

16 

28 

17 

c. Vitality 

. . 86 

3 

31 

13. 5 

d. Rhythm and fluency 

•• 7 

24 

4 

31.5 

e. Concise, simple language . . 

.. S4 

6 

17 

22 

f. Adult approach 

. 2 

35 

2 

34 

3. Literary techniques 

a. Dramatic techniques ... - 

• • 24 

IS 

32 

12 

b. Expository techniques .... 

.. 14 

19 

90 

I 

c. Figures of speech 

2 

35 

9 

27 

4, Format 

a- Type 

• • s 

27. 5 

6 

28.5 

b. Size of book 

I 

375 

0 

37 

c. Cover and illustrations . . . 

. 6 

25-5 

0 

37 

d. Footnotes, etc 

I 

37-5 

6 

28. s 

e. Sections and captions 

• • 3 

33 

4 

315 

f. Title of book 

• • 4 

30.S 

2 

34 

g. Make-up of book 

.. 8 

23 

0 

37 

C. Catching and holding interest 

I. Introduction 

a. Outline of plan ........ . 

• • 4 

30.S 

18 

20. s 

b. Awakening interest 

. . i6 

I7-S 

SI 

4 

2. Conclusion 

a. Summary of book 

■ • 13 

20 s 

33 

IX 

b. Outcome of reader interest , 

. . 27 

13 

49 

5 5 
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APPENDIX I C {Continued) 


Combined 

List of Critenon 

Criteria 

Frequency 

ral 

Rank 

Sec 

Frequency 

Knnb 

3. Relating subject to reader . 

46 

10 

55 

3 

4. Emotional appeal 

57 

4 

49 

5.5 

C. Use of famous author’s name . , . 

6 

25 5 


34 

D. Dealing with controversial issues 

I. Author’s display of viewpoint 

a. Non-partisan 

13 

20 s 

15 

23 

b. Fair partisan 

12 

22 

5 

30 

c. Prejudiced partisan 

29 

II 

12 

25 

2. Techniques of controversy 

a. Presentation of argument . . . 

28 

12 

48 

7 

b. Criticism 

2 

35 

14 

24 

c, Influence on reader 

47 

9 

31 

13.5 
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APPENDIX I D: COMBINED LIST OF CRITERIA 

WITH COMPONENT FREQUENCY RANKS COMBINED IN A SINGLE RANK ORDER 
INDICATING COMPARATIVE IMPORTANCE 


Criterion 

Judgm. 

Runki 

Reactions 

Rank! 

General 

Cnticisnis 

Rank* 

Expert 

Opinion 

Rank! 

Corn- 

Rank 

A. Content 

Historical allusions 

28* 

20.5 

17.S 

30* 

30 

Presentation of figures 

28 

10 

s 

6 

6.S 

Timeliness of information . . . 

7 

26 

27 -S 

IS 

IS 

Selection of information . . , . 

3 

2 

14 

30 

6.S 

3 

Quality of information 

S 

9 

2 

20 

B. Form 

Difficulty of terms 

28 

19 

30. s 

6 

20 

Difficulty of concepts 

9 

16 

2 

2 

Difficulty of abstractness .... 

28 

18 

30 s 

IS 

26.5 

Difficulty of condensation . . . 

28 

8 

7 -S 

9 s 

8 

Organization 

28 

IS 

7 -S 

30 

18.S 

Form and grammar 

28 

17 

16 

30 

2S 

Vitality 

8 

13.5 

3 

IS 

4 

Rhythm and fluency 

28 

31S 

24 

9-5 

28 

Concise, simple language 

28 

22 

6 

IS 

13 

Adult approach 

28 

34 

3 S 

IS 

36 

Dramatic techniques 

28 

12 

IS 

3 S 

10 

Expository techniques 

ii-S 

1 

19 

9 S 

s 

Figures of speech 

28 

27 

35 

30 

38 

Type 

16 

28 5 

27 S 

IS 

22 

Size of book 

10 

37 * 

37.S 

3 .S 

23 

Covet and illustrations 

4 

37 

2 S S 

6 

14 

Footnotes, index, etc 

I 3 S 

28 s 

37 S 

30 

3 S 

Sections and captions 

16 

3 I-S 

33 

20 

31 

Title of book 

I 

34 

30 . 5 

20 

21 

Make-up of book 

I 3 -S 

37 

23 

30 

32 

C. Holding reader interest 

Outline of plan 

28 

20.5 

30. s 

30 

33 

Awakening interest 

28 

4 

I 7 -S 

30 

17 

Summary of book 

28 

11 

20,5 

30 

24 

Outcome of reader interest . . . 

28 

S-S 

13 

IS 

II 

Relating subject to reader . . . 

6 

3 

10 

I 

I 

Emotional appeal 

16 

S-S 

4 

30 

9 

Use of famous author’s name . 

2 

34 

2 S-S 

30 

26.5 

D. Techniques of controversy 

Non-partisan viewpoint 

II. s 

23 

20 5 

9 'S 

12 

Fair partisan viewpoint 

28 

30 

22 

30 

34 

Prejudiced partisan viewpoint 

28 

2S 

II 

30 

29 

Presentation of argument — 

28 

7 

12 

30 

16 

Criticism 

28 

24 

35 

30 

37 

Influence on reader 

28 

*0 is 28 

13. S 

*0 is 37 

9 

30 

’"0 is 30 

18. s 


For frequencies upon which ranks are based, see (i) Appendix I B, (a) Appendix I C, (3) Appendix I A-a 
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APPENDIX II A 
INSTRUCTIONS TO READERS 

Tlie purpose of the first part of this study is to discover your interest in 
various pamphlets, your f^rsorml leaction to them. There are several series 
of three pamphlets. Each series is upon one subject which is presented dif- 
ferently by each pamphlet. You will read one series at a time, will consider 
each pamphlet separately first, and then in comparison with others. 

Your personal feelings, as expressed in the answer to “Why?” or “Reason” 
are even more important than your ratings. Please make them as specific as 
possible. But be brief in your comments. You will be asked to raU each pam- 
phlet according to your interest in different parts of it. This is explained in 
detail on the Section Rating Sheet. Then you will rank the three pamphlets 
in order of your preference for them, as directed below. 

Use this sheet only as a guide. Make out a separate plain sheet for each 
series, in order to answer questions I and III. Write your answer to II on the 
back of each Section Rating Sheet. You need not copy questions; write the 
outline number thus' I-A; III-B-4. And instead of writing out the full name 
of each pamphlet in your answers, use the key letter in the upper right corner 
of the cover. 

Do Mch Stef in the oidei directed beloze, 

I. Take a couple of minutes to examine the three pamphlets, and record 

your snap judgment on the following; 

A. Which would you like to read first? Why? 

B. Which title and cover do you prefer? Rank in order of preference. 

II. Study each pamphlet carefully according to the outline. First, read it 

through rapidly at the rate you would ordinarily devote to it. If you 

would usually skip part of it, tell where and why. 

A. Do you think it (i) very interesting, (2) interesting, (3) dull? 

B. Please make specific criticisms (both favorable and unfavorable) on the 
way the author has presented his material, his style, the way they affect 
your interest. 

C. Read through the pamphlet again, rating each section on the prepared 

III. Compare all the pamphlets in the series. 

A. Which is most interesting to you? Why? Rank in order of prefer- 

B. Compare all three pamphlets on each of the following points, ranking 
them with i meaning most interesting to you in that respect and 3 
least. If you are in doubt as to ranking, if all seem on a level or none 
is at all satisfactory, comment on this and do not rank. Remember to 
rank the three pamphlets on each of the six points. 

I. Which is (most, medium, least) inviting in the kind and size of 
print used, in appearance of page? 
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2. Which explains tables and numbers clearly? 

3. Which is well written? 

4. Which is simple and easy to understand? 

5. Which meets your own needs and understanding of the subject as 
far as its explanations and technical terms go? 

6. Which seems to touch your personal questions or experience? 

APPENDIX II B 
SECTION RATING 

This is your direction sheet. Do not use it for rating, but keep it for 
reference. 

Directions: In the lines below write (a) the tide of the pamphlet, (b) 
the code letter in the upper right corner of its cover, (c) You will be rating 
three pamphlets in this series. Over “Number” mark i for the first one ex- 
amined, 2 for the second, and 3 for the third, (d) Write your own name. 

Read each section, and then make a check in the column of Interest opposite 
the section number to show your opinion of it. Write the reason for your 
judgment under reason. That remark should always indicate the section it 
is explaining, by writing the section number before it if it does not correspond 
to the number in the margin. 

If the same reason occurs again, do not rewrite it. Place the number of this 
repeated section opposite the reason as stated the first time, in the right hand 
column under same. But analyse your opinions; do not use the last column 
often. 

If the pamphlet has more paragraphs than are numbered on the sheet, use 
the opposite side of the paper. The other side will be used also for answers 
to question II on your other sheet of directions. 

Under interest, the abbreviations are V for Very interesting, R for Rather 
interesting, M for Medium, S for Slightly interesting, D for Dull. 


Pamphlet Letter Number Name 

INTEREST REASON SAME 

V. R. M, S. D. 


Sect. 

No. 

2 

3 
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APPENDIX II C 
EXPERIMENTAL CHECK LIST 

Your -name Date 

Name of the book you read 

Instructions 

Read this book all the way through. Then mark what you think of it on 
these sheets. There are six groups of reasons for you to mark. If you have 
any other reasons that are not given, write them on the last page. 

On the left side of each page are reasons for liking the book, and on the 
right are things that you don’t like about the book. Read I on the left side, 
under “Things I Like about the Beginning of this Book,” and then i on the 
right side under “Things I do not Like about the Beginning of the Book.” 
Check the reason that tells the way you feel about i (like this: . .X . i). 
Then read 2 on the left and 2 on the right, and so on for every reason. 

You do not need to check a reason if neither side is true about your book. 
For instance, it asks you in B-2 whether you understand the history in the 
book. If there is no history in the one you have read, then you can’t check 
that reason. 

If two reasons are connected by the word or, they are opposites and you can 
not check both of them. Look at A-2 and 3 just below. These questions ask 
you about your knowledge of the subject. If you don’t know about it, you 
will check 2 on either the right or the left half of the paper. If you do know 
about it you will check 3. You could not check both 2 and 3. Think in the 
same way about all other reasons joined by the word or. 

Be sure to read and to think about every single reason printed here. 

A. Things I like about the beginning A. Things I DO not like about the BE- 
of this book. ginning of this book. 

. ..1. It tells clearly why this book . l. It does not tell why this book 
was written. was written. 

. a. I know nothing about this sub- ... z. I know nothing about this sub- 
ject and want to find out some- ject and do not care to. 

thing. 

.. 3. I know something about this ..3. I have heard too much about 

subject, but would like to learn this already, 

more. 

. . .4. The beginning makes me think 4. The beginning makes me think 

this book will be neither too this book will be hard to' under- 

hard nor too easy, that it will stand, 

be just right. 

. 5. The beginning seems too easy. 
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B. Things I like about the main part 
of this book. 

1. People and newspapers are talk- 
ing about this subject now. 

2. Enough history about the sub- 
ject is given for me to under- 
stand what I read. 

. 3. The writer does not take one 
side against the other; he tells 
me about both sides of the prob- 

4. The writer talks for one side, 
but seems to tell the truth about 
the other. 

. .5. There are charts, figures, and 
tables which are interesting and 
easy to read. 

. .6, There are no charts or figures, 

which makes me like the book 
better. 

C. Things I like about the ending of 
this book. 

. . I. The writer brings together in 
a short clear way all of the 
things he was writing about. 

. , ,2. The ending is written in a way 
that makes me feel that I agree 
with the writer. 

... .3. Though I do not agree with the 
writer, I think he has tried to 
be truthful. 

. 4. The ending makes me Interested 

in reading more about the sub- 
ject. 

, 5. The ending makes me feel as 

if I know all .about this subject 
from what I have read. 

D. Things I LIKE about the writer of 
this book. 

... I. He uses words I can understand. 

. . . 2. He tells good stories and ex- 
amples to bring out what he 
wants to say. 


ZI9 

B. Things I DO NOT LIKE about the 
MAIN PART of this book. 

1. This subject is too old. Nobody 
talks about it now. 

. . 2. Not enough history is given for 

me to understand the subject 
completely. 

3. The writer teUs only one side 
of the story; he does not give 
the other. 


... 4. The writer takes one side of the 
story; and I do not think he is 
telling the truth about the other. 

. ..5. There are charts and tables of 
figures which are uninteresting 
and hard to read. 

... 6. I wish there were charts and fig- 
ures, to make the book more 
interesting. 

C. Things 1 DO NOT LIKE about the 
ENDING of this book. 

....1. The writer does not bring to- 
gether clearly what he was talk- 
ing about. 

. .2. The ending is written in a way 

that makes me feel that 1 do 
not agree with the writer. 

3. I do not think the writer has 

been fair to the other side. 

.... 4. I do not want to read more 
about this. 

or 

. 5. 1 feel as if I learned very little 
about this subject from what I 
read. 

D. Things I DO NOT like about the 
WRITER of this book. 

... I. He uses many words I do not 
understand, 

... 2. He does not tell any stories; the 
book would have been more in- 
teresting with them. 
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. . . 3. He does not need stories or ex- 
amples to make the book inter- 

.... 4. He makes me feel as if this 
problem mattered to me. 

. . , 5. He -writes just enough about the 
subject. 


E. Things I like about the book as a 

WHOLE. 

. . .1. It is made of good paper. 

, , a. It is well printed, with type 
clear and easy to read. 

....3. It is broken into just enough 
chapters or sections to make it 
easy and interesting to read. 


.5. Its section titles are good and 
helpful. 

. 6, The cover attracted my atten- 


....3. He gives silly stories and ex- 
amples. 

. 4, He does not leave me feeling 

that this makes any difference 

5. He does not write enough. 

.... 6. He writes too much and says 
little. 

E. Things I DO NOT LIKE about the 
BOOK AS A WHOLE. 

... .1. It is made of poor paper. 

. . 2. It is badly printed, with type 
hard to read. 

3. It is broken into too many sec- 


. 4. It is broken into too few sec- 

5. The titles for the section are 
not helpful. 

. 6. The cover has no life, and did 
not attract me. 

. 7. The title of the book did not 
interest me. 


.... 7. The title of the book made me 
want to read it. 

How did you like this book? (Place a check beside the words that best tell 
what you think about it on the whole.) 

It was . Very interesting. . Interesting. Not at all interesting. 

If there were any reasons why you liked or did not like this book, which did 
not appear on these sheets, tell them briefly below and on the other side in your 
own words: 


APPENDIX II D 
PERSONAL INFORMATION 

Name . . . . Date . . . . . 

Sex . Married?. Age. . . . 

Check the grade nearest to the one when you left school: 8 

high school . 5 2 years college .... College grad , . . Degrees 


Have you attended any classes smee you left school? 

What kind, and how long? 

Have you taken courses in economics? Sociology? ... . Political 

Science? . . . . 
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WJiat newspaper do you read regularly? 

What are youi favorite magazines? 

How many books have you read in the past month? . ... Name one: 

Which church do you usually attend? Catholic .... ; Protestant . . ; 

Jewish . , None 

What are your political tendencies? Republican ; Democrat .... ; 

Socialist. . . ; Communist . ; Split ticket 

What position were you trained for? . What jobs have you held? 

Have you read any pamphlets during the past year? 


You will be reading pamphlets on four different topics. Place a check after 
any topics that you would ordinarily be interested in reading about: 

Unemployment . . . . 5 War and peace , Changing govern- 
ment ; Stability of money . 

Which would you like to read first? Rank the four in the order of their interest 
to you, with r as most interesting, and 5 as least interesting: 

Unemployment .... ; War and peace . i Changing govern- 
ment . j Stability of money . . . . 

APPENDIX II E 

LETTER TO ASCERTAIN METHODS OF SELECTION AND 
EVALUATION OF CURRICULUM MATERIAL, ADDRESSED 
TO TYPICAL ADULT EDUCATORS 

March, 1934 

Attention of. 

Gentlemen: 

I am planning to organize classes of a non-professional nature for adults and 
should appreciate any information you may have about teaching material, with 
reference to the following points: 

1. Do you prepare your own material for courses or do you depend entirely 
on already published materials? 

2. If you prepare your own material, what criteria do you use in evaluating 
its success in practice? 

3. If you use already published material, how do you determine its suc- 
cess with your groups? 

4. Can you suggest the names of persons or organizations or classes doing 
experimental or exploratory work with curriculum material for adults 
on any educational level? 

Yours truly. 


(Signed) William Kolodney 
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APPENDIX III A 


COMPONENTS OF PAMPHLET PREFERENCE INDEX 
A. General rating of twelve famfhlets for degree of interest 



Nnml, 

ler of Votes 







Pamphlet 

Very 

ntmr 

Dull 

Weighted Votes 

Weighted 

Num- 

Average 

Rank 


ti) 

(a) 

(3) 

(X) 

(s) (3) 

VoU» 

Votes 



Unemp’m’t 










B 

S 

24 

4 

S 

48 12 

6S 

33 

1.98 

9 

C 

ir 

16 

7 

II 

32 21 

64 

34 

1.88 

4 

D 

II 

IS 

3 

II 

30 9 

SO 

29 

1.72 


Peace 









E 

7 

18 

7 

7 

36 21 

64 

32 

2 00 

10. 5 

F 

9 

24 

2 

9 

48 6 

63 

35 

1.80 

3 

G * ... 
Government 

10 

18 


■ 10 

36 6 

52 

30 

1-73 

2 

H 

8 

21 

7 

8 

42 21 

71 

36 

1-97 

7-5 

I 

6 

2S 

4 

6 

SO 12 

68 

35 

1.94 

6 

K 

Money 

10 

16 

9 

10 

32 27 

69 

35 

I 97 

7-5 

0 

4 

29 

4 

4 

58 12 

74 

37 

2.00 

10. 5 

P 

10 

18 

7 

10 

36 21 

67 

35 

1. 91 

5 

Q 

4 

23 

8 

4 

46 24 

74 

35 

2. II 

12 

B. 

Section rating of twelve famfhlets by forty-five readers 



Numb, 

:r of Votes on Degree of Interest 

Sumtof 




Pamphlet 

{!)* 

for All 
(2) 

Pamphlet Section! 
(3) (4) 

(3) 

AwSges 

Sections 

Rating 

Rank 

Unemp’m’t 










B 

140 

251 

242 

IS4 

81 

52 32 

19 

2-75 

9 

C 

lOI 

I2I 

102 

4S 

30 

22 48 

9 

2.50 

4 

D 

130 

IIS 

107 

54 

27 

24-05 

10 

2 41 

2 

Peace 










E 

163 

196 

228 

157 

88 

52.80 

19 

2 78 

II 

F 

317 

34S 

354 

188 

IIS 

74-45 

29 

2-57 

7 

G 

Government 

102 

1 14 

72 

48 

23 

19 00 

8 

•2.38 

I 

H 

204 

216 

294 

136 

124 

58.21 

21 

2.77 

10 

I 

182 

346 

247 

104 

91 

56.38 

22 

2.56 

6 

K 

Money 

220 

207 

197 

95 

83 

45.60 

18 

2 53 

5 

0 

78 

202 

187 

119 

71 

40 29 

14 

2,88 

12 

p 

105 

109 

132 

67 

18 

24.81 

10 

2.48 

3 

Q 

109 

225 

186 

95 

59 

39-59 

IS 

2.64 

8 


* (i) indicates a “very interesting” section, (s) a “dutl” one. 

t Votes in each column were weighted accordi^ to the number at its head Weights in five columns were 
added and sum was divided by number of voters. The sum of all the resulting section averages is given in 
the siitth column 
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APPENDIX III B 

THREE DIFFERENT GROUPS’ FREQUENCY OF CITING 
STANDARDS USED IN THEIR EVALUATION OF 
EIGHT PAMPHLETS 


Classification of Standards According 
to the Check List 

C.WA. 

Fre- ^ 

Harlem 

. Rank 

quencyr 

quencyt 

Rank 

A. Introduction 

I. States purpose . 

39 

II 

45 

I 

23 

9 

2. New information 

3 

17 

17 

Z 2 

3 

24. 3 

3. Interesting old infor- 

mation 

SI 

9 

39 

II 

29 

i-S 

4. Proper difficulty .... 

9 

16 

44 

2.3 


12 

B. Body 

I. Timely 

54 

8 

42 

6 

29 

i.S 

2, Sufficient history . . . 

60 

7 

30 

17 

23 

II 

3. Impartial 

38 

12 

35 

14 

9 

20.3 

4. Fair partisan 

35 

13 

16 

23 

16 

IS 

5. Interesting figures . . 

85 

5 

32 

16 

10 

19 

or 6 . Lack of figures 


18.S 

II 

25 

3 

24 3 

C. Conclusion 

I, Clear summary 

112 

4 

43 

4.5 

25 

9 

2. Reader agrees 

129 

3 

40 

10 

21 

13.5 

or 3. Reader disagrees, but 

author was fair . . 

I 

18.5 

5 

26 

4 

23 

4. Reader desires to learn 

more 

43 

10 

41 

8 

26 

6 

3. Reader learned much 

13 

15 

IS 

24 

0 

26 

D. Author 

I. Used clear words . . . 

230 

I 

44 

2.5 

27 

3.5 

2. Used good examples . 

l8S 

2 

34 

IS 

IS 

16 

3. Needed no examples . 

0 

20 

18 

21 

5 

22 

4. Made material seem 

important . ... 

31 

14 

41 

8 

27 

3-5 

S. Gave enough informa- 

tion 

67 

6 

24 

19 

14 

17 

E. Mechanics 

1. Good paper 

2. Well printed 

3. Well sectioned 

4. Good section titles . . 
3. Attractive cover .... 
6. Interesting title 



28 

43 

37 

38 

21 

41 

18 

4-5 

13 

12 

20 

8 

12 

25 

21 

26 

9 

26 

18 

9 

13-5 

6 

20.3 

6 


* Frequency of section criticisms when classified under Cheek List topics 
t Frequency of checking item on Check List 
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APPENDIX III C 

COMPARISON OF IMPORTANCE RANKS OF MAIN STANDARDS, 
DERIVED FROM GENERAL AND FROM SECTION CRITICISMS 
A. Comfarison of two methods of criticism for one famfhlet, I 


Cla=,Cntmon 

quency 

Rank 

General 

qu^^y 

Difference 

between 

Ranks 

A I. Historical allusions . . 

12 

12 

J 

10. s 

1-5 

2. Presentation of figures . . 

20 

9 

0 

13 

-4 

3. Treatment of subject .... 

84 

2 

25 

3 


B I. Difficulty 

80 

3 

26 


I 

2. Literary quality 

44 

5 

30 

I 

4 

3. Literary techniques 

los 

I 

II 

5 

-4 

4. Format 

4 

13 

0 

13 


C I. Introduction 

24 

7 

I 

10. s 

-3.5 

2. Conclusion 

18 

10 

2 

9 


3. Catching reader interest .. 

14 

II 

9 

6 

5 

4. Emotional appeal 

22 

8 

4 

8 


S. Use of famous author . . 

0 

14 


13 

I 

D I. Author’s viewpoint 

30 

6 

19 

4 

2 

2. Techniques of controversy 

57 

4 

5 

7 

-3 

Rho is . 

77- 

P.E. IS 

.078 



B. Comparison of two methods 

f ciiticism for twelve pamphlets 



Gene 

al 

DiSerenc e 

Class Criteri!,n 

Fre- 


Fre- 




quency 


quency 


Kanks 


A I. Historical allusions 22 

2. Presentation of figures , 66 

3. Treatment of subject . . . 122 

B I. Difficulty IIS 

2. Literary quality 107 

3. Literary techniques .. .. 177 

4. Format 24 

C I. Introduction 68 

2. Conclusion S® 

3. Catching reader interest . . 64 

4. Emotional appeal 56 

5. Use of famous author 2 

D I. Author’s viewpoint S3 


2. Techniques of controversy ill 
Rho is .68. 


13 16 13 

7 S6 6 

2 117 3 

3 154 2 

5 220 I 

I 41 9 

12 28 II 

6 20 12 

II 40 10 

8 46 8 

9 57 S 

14 6 14 

10 S4 7 

4 77 4 

P.E. is .101. 
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APPENDIX III D 

SKETCH OF THE CONTENT OF THE EXPERIMENTAL 
PAMPHLETS, AND OF THE AUTHORS’ METHODS OF 
RELATING SUBJECTS TO READERS 

The contents of each pamphlet were described by from two to four assist- 
ants. Methods employed by the writer to catch and to hold reader interest 
were described. The assistants’ analysis form the basis for the following sum- 
maries of content and human interest techniques. Pamphlets D and O are 
omitted here, since they are treated in detail in Chapter V. 

Series; Unemfloyment 

Pam-phlet B — "Unemployment Relief and Public Works” 

Content: After a brief statement of the effects of the depression and of the task 
ahead of the government, the pamphlet attempts to give in an unbiassed 
manner a picture of the Federal Emergency Relief Administration. It deals 
with the administrative oiganization, the possible permanence of its function- 
ing, and the projects undertaken by the F.E.R.A. Most space is given to 
the Civilian Conservation Corps and the Tennessee Valley Authority. 
Human interest techniques. The author strives to present fair, unbiassed state- 
ments covering both sides of important arguments. There is a generous 
supply of background and facts which are calculated to hold the reader’s 
interest. 

Pamfhlet C — “Uncle Sam and Unemployment” 

Content; The writer gives a short history of the depression as it affects a par- 
ticular family. She traces the members of the family through their experi- 
ences with different phases of the Recovery Administration, showing the 
positive effects of the various acts since 1932. (The data are the same as 
those given without dramatization in Pamphlet B.) 

Human interest techniques; Most of the information is presented in narrative 
form, through the medium of a typical unemployed family. The topic is 
introduced by questions directed at the reader, tying the problem to his own 
experience and immediately making him part of the conditions he reads 
about. The language is simple. A cartoon on the first page lends Interest. 

Series; War and. Peace 
Pamphlet E — “Who Pays for War?” 

Content; This pamphlet represents an attempt to answer the question raised in 
the title. It treats direct cost of armaments, reparations and war debts, and 
indirect effects of pensions, cost to survivors, effect on industry, finance and 
business. 

Human interest techniques: Facts are presented in graphic and dramatic forms, 
showing the cost and wastefulness of war. This, with an account of the 
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loss of life and the suffering, is material appealing to human interest. The 
writer makes it apparent that everyone suffers from war, that those who 
gain a victory are not spared. 

Pamphlet F — “The Fight Against War” 

Content; This is a collection of extracts from pacifistic speeches and writings 
of Albert Einstein. Eacli extract is prefaced by an editorial note stating time, 
place, and occasion of utterance. Not only does the author reveal his own 
reasons for his anti-war feeling, but he also shows what procedures should 
be taken by others who would join in pacifistic movements. 

Human interest techniques: The technique in this case is that of the editor 
rather than of the author. He selected the most important points from ad- 
dresses, personal letters, and interviews. Each is headed by a caption carry- 
ing the organizing thought and is prefaced by a note. The suggestion of 
brevity given by asterisks between excerpts is another technique for catching 
the eye. 

Pamfhlet G — “War Resistance” 

Content: These background arguments are defended: so far as war is con- 
cerned human nature can be changed; defensive war leads to aggressive war. 
The cost of preparedness is discussed, as are other causes of war. The paci- 
ficist’s stand is given, that war resistance is the only effective instiument 
against war. “No war is inevitable or justified.” 

Human interest techniques; Arguments of opposing views are stated fairly, 
then answered. The writing is dramatic and emotional at strategic points. 
The author plays upon the horror, the suffering, and the stupidity of war. 

Series’ Changing Governments 

Pamphlet H — “Roosevelt and the Constitution” 

Content; H presents an analysis of Roosevelt’s actions in office. It deals with 
the questions: is he a dictator? does he violate the constitution? is he “mak- 
ing” the laws? will the Constitution suffer from the New Deal^ It con- 
trasts the viewpoint of liberals with those conseivatives who say that the 
Constitution should be followed literally. 

Human interest techniques: Since this was written for high school students. It 
has a dramatic setting of a high school class that calls a professor in to an- 
swer its questions. The content is in quotation maiks. Each section is in- 
troduced with a question or topic sentence, which is elaborated with concrete 
examples in the rest of the paragraph. 

Pamphlet 1 — “Modern Economic Systems” 

Content: Four economic systems are analyzed: Capitalism, Communism, Fas- 
cism, Socialism. The meaning and the record of each is given, concluding 
each section with a discussion of “weak spots.” At the end faith is ex- 
pressed in the “New Capitalism.” 

Human interest techniques. An analogy at the beginning between Capitalism 
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and a car that will not run catches immediate attention. The procedure then 
is explicit definitions, statement of the most pertinent facts, and comparison 
of the various systems to show similarities and differences, as well as weak 
points. 

Pamfhlet K — “The Yankee Primer” 

Content. The writer attempts to show that democracy has never existed in the 
United States. In its stead the Hamilton Ideal, which since its beginning 
has supported striving for profit, has subjected the mass of people to slavery. 
The writer calls upon the workers to establish what he calls an Industrial 
Democracy. His methods for attaining this new society, as outlined, are 
socialistic. 

Human intet est techniques: The technique is one of dramatic, colloquial, simple 
language. The text is enlivened by colorful description and a generous use 
of simple comparisons and dramatic explanations. The familiar style causes 
violent opinions, either in favor of, or opposed to, the pamphlet. It is a style 
popular with labor leaders and organizers. 

Senes: Unstable Money 

Pamfhlet P — “Unstable Money” 

Content: The writer establishes the point that it is impossible to discover a 
monetary commodity which will remain stable in price. Money, like other 
commodities, is liable to changes in price owing to the demand for it. There 
is no simple way of stabilizing money. 

Human interest techniques: A literary, conversational style is combined with 
talking like an expert, picking holes in other experts’ theories and developing 
one. 

Pamfhlet Q — “Inflation and Your Money” 

Content: A background is built of the history of currency and its place in a 
monetary system, of banks and the function of the Federal Reserve System, 
of the international gold standard. Inflation is explained, its history in 
America and other countries pointed out, and its dangers made clear. The 
author favors sound currency. 

Human interest techniques: The author writes simply and clearly, frequently 
summarizing, giving illustrations. He assumes an intelligent reader, who 
will agree with his common sense viewpoint, yet dramatizes his information 
so cleverly that an unintelligent person would agree. 
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READER RANKING OF SIX QUALITIES IN THE TWELVE 
PAMPHLETS 

Mean ranking in series of three on — * 


VITA 


Margaret Charters Lyon was born in Columbia, Missouri, April 
30, 1909. She is the daughter of Werrett Wallace and Jessie Allen 
Charters. The first two years of her undergraduate college work 
were carried on at the University of Wisconsin; she received a 
Bachelor of Arts degree at Ohio State University in 1930. After 
several years of graduate work at Teachers College, Columbia Uni- 
versity, she was awarded the Master of Arts degree and became a 
candidate for the Doctor of Philosophy degree. Membership in Phi 
Beta Kappa and Kappa Delta PI was conferred upon her at Ohio 
State University and Columbia University respectively. She was a 
national officer of the honorary activities sorority. Mortar Board. In 
1935 she was married to Ralph Muse Lyon. In 1936 she became as- 
sociate educational director of the Greenville County Council for 
Community Development, and associate professor of, education at 
Furman University. 



